2025-11-28T18:22:25.652252Z node 1 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[1:7577843352840752279:2081];send_to=[0:7307199536658146131:7762515];
2025-11-28T18:22:25.652299Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
2025-11-28T18:22:25.659109Z node 1 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-11-28T18:22:25.660269Z node 2 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[2:7577843354891841157:2176];send_to=[0:7307199536658146131:7762515];
2025-11-28T18:22:25.660370Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
test_client.cpp: SetPath # /home/runner/.ya/build/build_root/1xri/00022d/r3tmp/tmplw3JXu/pdisk_1.dat
2025-11-28T18:22:25.661970Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-11-28T18:22:25.694267Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-11-28T18:22:25.708881Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-11-28T18:22:25.710914Z node 1 :IMPORT WARN: schemeshard_import.cpp:423: Table profiles were not loaded
TServer::EnableGrpc on GrpcPort 2670, node 1
2025-11-28T18:22:25.719872Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:190: distributable config is empty, broken or outdated, will use file: /home/runner/.ya/build/build_root/1xri/00022d/r3tmp/yandexTcJLXH.tmp
2025-11-28T18:22:25.719890Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:196: will try to initialize from file: /home/runner/.ya/build/build_root/1xri/00022d/r3tmp/yandexTcJLXH.tmp
2025-11-28T18:22:25.719966Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:202: successfully initialized from file: /home/runner/.ya/build/build_root/1xri/00022d/r3tmp/yandexTcJLXH.tmp
2025-11-28T18:22:25.720002Z node 1 :NET_CLASSIFIER ERROR: net_classifier.cpp:228: got bad distributable configuration
2025-11-28T18:22:25.723191Z INFO: TTestServer started on Port 12753 GrpcPort 2670
TClient is connected to server localhost:12753
PQClient connected to localhost:2670
WaitRootIsUp 'Root'...
TClient::Ls request: Root
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "Root"
    PathId: 1
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 1
    CreateStep: 0
    ParentPathId: 1
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 1
      SubDomainVersion: 0
      SecurityStateVersion: 0
    }
    ChildrenExist: false
  }
  Children {
    Name: ".sys"
    PathId: 18446744073709551615
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 0
    CreateStep: 0
    ParentPathId: 18446744073709551615
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 0
      Pl...
(TRUNCATED)
WaitRootIsUp 'Root' success.
2025-11-28T18:22:25.753725Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-11-28T18:22:25.753758Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-11-28T18:22:25.755597Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-11-28T18:22:25.763860Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpAlterSubDomain, opId: 281474976715657:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_alter_subdomain.cpp:311)
2025-11-28T18:22:25.764238Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-11-28T18:22:25.764268Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-11-28T18:22:25.766881Z node 1 :HIVE WARN: hive_impl.cpp:811: HIVE#72057594037968897 Handle TEvInterconnect::TEvNodeConnected, NodeId 2 Cookie 2
2025-11-28T18:22:25.767176Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Connecting -> Connected
waiting...
waiting...
2025-11-28T18:22:25.780364Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976715659:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
waiting...
waiting...
waiting...
2025-11-28T18:22:25.904398Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-11-28T18:22:25.962556Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-11-28T18:22:26.040121Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7577843359186808722:2303], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-28T18:22:26.040148Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7577843359186808693:2300], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-28T18:22:26.040162Z node 2 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-28T18:22:26.040293Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7577843359186808729:2305], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-28T18:22:26.040316Z node 2 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-28T18:22:26.042748Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateResourcePool, opId: 281474976720657:3, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:179)
2025-11-28T18:22:26.048307Z node 2 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [2:7577843359186808728:2304], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976720657 completed, doublechecking }
2025-11-28T18:22:26.137648Z node 2 :TX_PROXY ERROR: schemereq.cpp:593: Actor# [2:7577843359186808757:2172] txid# 281474976720658, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 9], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-11-28T18:22:26.143016Z node 1 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:625: Compilation failed, self: [1:7577843357135720642:2331], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-11-28T18:22:26.143135Z node 1 :KQP_SESSION WARN: kqp_session_actor.cpp:2542: SessionId: ydb://session/3?node_id=1&id=ZjE0OTdmOTUtYzg5MDBkOWUtZDI5OTMyZWEtMTAxMzhmM2Q=, ActorId: [1:7577843357135720600:2323], ActorState: ExecuteState, TraceId: 01kb5v7fsfdck6hhkr9jz6aj5d, ReplyQueryCompileError, status: SCHEME_ERROR, issues: { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } }, remove tx with tx_id: 
2025-11-28T18:22:26.143126Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976715662:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
2025-11-28T18:22:26.143486Z node 2 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:625: Compilation failed, self: [2:7577843359186808772:2309], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-11-28T18:22:26.143573Z node 1 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-11-28T18:22:26.143843Z node 2 :KQP_SESSION WARN: kqp_session_actor.cpp:2542: SessionId: ydb://session/3?node_id=2&id=M2UwMWJkMDktMTlkODQ5MS01ODJjNzc2MC0xOWZkODE3Ng==, ActorId: [2:7577843359186808688:2298], ActorState: ExecuteState, TraceId: 01kb5v7fqq06re7htnth9sd4tk, ReplyQueryCompileError, status: SCHEME_ERROR, issues: { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } }, remove tx with tx_id: 
2025-11-28T18:22:26.143943Z node 2 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-11-28T18:22:26.156615Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976715663:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
2025-11-28T18:22:26.179307Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976715664:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
=== Init DC: UPSERT INTO `/Root/PQ/Config/V2/Cluster` (name, balancer, local, enabled, weight) VALUES ("dc1", "localhost", true, true, 1000), ("dc2", "dc2.logbroker.yandex.net", false, true, 1000);
=== CheckClustersList. Subcribe to ClusterTracker from [1:7577843357135721077:3051] 
2025-11-28T18:22:26.654546Z node 1 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-11-28T18:22:26.661704Z node 2 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-11-28T18:22:30.652270Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[1:7577843352840752279:2081];send_to=[0:7307199536658146131:7762515];
2025-11-28T18:22:30.652315Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
2025-11-28T18:22:30.660449Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[2:7577843354891841157:2176];send_to=[0:7307199536658146131:7762515];
2025-11-28T18:22:30.660499Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
=== CheckClustersList. Ok
PQ Client: create topic: rt3.dc1--topic1 with 1 partitions
CallPersQueueGRPC request to localhost:2670
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--topic1"
  }
}
2025-11-28T18:22:31.410434Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--topic1, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:2670
MetaRequest {
  CmdCreateTopic {
    Topic: "rt3.dc1--topic1"
    NumPartitions: 1
    Config {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
  }
}
CallPersQueueGRPC response:
Status: 129
ProxyErrorCode: 53
SchemeStatus: 1
FlatTxId {
  TxId: 281474976715671
  SchemeShardTabletId: 72057594046644480
  PathId: 13
}
ErrorCode: OK
AddTopic: rt3.dc1--topic1
===Run query:``DECLARE $version as Int64; DECLARE $path AS Utf8; DECLARE $cluster as Utf8; UPSERT INTO `/Root/PQ/Config/V2/Topics` (path, dc) VALUES ($path, $cluster); UPSERT INTO `/Root/PQ/Config/V2/Versions` (name, version) VALUES ("Topics", $version);`` with topic = topic1, dc = dc1
2025-11-28T18:22:31.418164Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1656: [72075186224037893][] pipe [1:7577843378610557988:3288] connected; active server actors: 1
2025-11-28T18:22:31.418311Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1520: [72075186224037893][rt3.dc1--topic1] updating configuration. Deleted partitions []. Added partitions [0]
2025-11-28T18:22:31.418409Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:942: [72075186224037893][rt3.dc1--topic1] Discovered subdomain [OwnerId: 72057594046644480, LocalPathId: 1] state, outOfSpace = 0 at RB 72075186224037893
2025-11-28T18:22:31.418770Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:140: [72075186224037893][rt3.dc1--topic1] BALANCER INIT DONE for rt3.dc1--topic1: (0, 72075186224037892) 
2025-11-28T18:22:31.421785Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:404: [72075186224037893][rt3.dc1--topic1] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1
2025-11-28T18:22:31.420870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2959: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-11-28T18:22:31.421537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2991: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-11-28T18:22:31.421597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:787: [PQ: 72075186224037892] no config, start with empty partitions and default config
2025-11-28T18:22:31.421603Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:913: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-11-28T18:22:31.421614Z node 2 :PERSQUEUE INFO: pq_impl.cpp:612: [PQ: 72075186224037892] doesn't have tx writes info
2025-11-28T18:22:31.421735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610557987:3287], now have 1 active actors on pipe
2025-11-28T18:22:31.421739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558010:2454], now have 1 active actors on pipe
2025-11-28T18:22:31.421921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3563: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-28T18:22:31.422552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-28T18:22:31.424375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558046:3335], now have 1 active actors on pipe
2025-11-28T18:22:31.424583Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:84: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitConfigStep
2025-11-28T18:22:31.424611Z node 2 :PERSQUEUE DEBUG: partition.cpp:1315: [72075186224037892][Partition][0][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-11-28T18:22:31.424666Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:84: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-11-28T18:22:31.424736Z node 2 :PERSQUEUE INFO: partition_init.cpp:1177: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7577843380661645517:2363]
2025-11-28T18:22:31.425181Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:84: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitFieldsStep
2025-11-28T18:22:31.425189Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:64: [rt3.dc1--topic1:0:Initializer] Initializing completed.
2025-11-28T18:22:31.425193Z node 2 :PERSQUEUE INFO: partition.cpp:707: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--topic1' partition 0 generation 1 [2:7577843380661645517:2363]
2025-11-28T18:22:31.425198Z node 2 :PERSQUEUE DEBUG: partition.cpp:727: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--topic1 partitition 0 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-11-28T18:22:31.425203Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:107: [72075186224037892][Partition][0][StateIdle] Initializing MLP Consumers: 0
2025-11-28T18:22:31.425411Z node 2 :PERSQUEUE DEBUG: partition.cpp:4408: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 1
2025-11-28T18:22:31.425426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.425443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2392: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TTransaction[ProposeConfig])
2025-11-28T18:22:31.425454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.425462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.425464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.425466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.425483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.425490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.425491Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.425493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.425494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.425501Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:930: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-11-28T18:22:31.425508Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:70: [72075186224037892][Partition][0][StateIdle] Process MLP pending events. Count 0
2025-11-28T18:22:31.425580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3563: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-28T18:22:31.425608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.426371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-28T18:22:31.426411Z node 2 :PERSQUEUE DEBUG: partition.cpp:1420: [72075186224037892][Partition][0][StateIdle] Handle TEvPQ::TEvTxCommit Step 1764354151469, TxId 281474976715671
2025-11-28T18:22:31.426416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.426419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.426421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.426427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2456: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TTransaction[ProposeConfig])
2025-11-28T18:22:31.426444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.426454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.426458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.426519Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.427206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.427244Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:107: [72075186224037892][Partition][0][StateIdle] Initializing MLP Consumers: 0
2025-11-28T18:22:31.427274Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.427287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.427290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.427292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.427294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.427296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.427302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.427416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:398: [PQ: 72075186224037892] Apply new config PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--topic1" Version: 0 LocalDC: true Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } TopicPath: "/Root/PQ/rt3.dc1--topic1" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Version: 0 Generation: 0 } MonitoringProjectId: ""
2025-11-28T18:22:31.427438Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:913: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-11-28T18:22:31.427482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3563: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-28T18:22:31.428010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-28T18:22:31.428042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3563: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-28T18:22:31.428694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
===Query complete
TClient::Ls request: /Root/PQ/rt3.dc1--topic1
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--topic1"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976715671
    CreateStep: 1764354151469
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--topic1"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 600000...
(TRUNCATED)
GetTopicVersionFromPath:  record Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--topic1"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976715671
    CreateStep: 1764354151469
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--topic1"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      YdbDatabasePath: "/Root"
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
    Partitions {
      PartitionId: 0
      TabletId: 72075186224037892
      Status: Active
    }
    AlterVersion: 1
    BalancerTabletID: 72075186224037893
    NextPartitionId: 1
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 1
      PlanResolution: 7
      Coordinators: 72057594046316545
      TimeCastBucketsPerMediator: 2
      Mediators: 72057594046382081
    }
    DomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    PathsInside: 12
    PathsLimit: 10000
    ShardsInside: 6
    ShardsLimit: 200000
    ResourcesDomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    DiskSpaceUsage {
      Tables {
        TotalSize: 0
        DataSize: 0
        IndexSize: 0
      }
      Topics {
        ReserveSize: 0
        AccountSize: 0
        DataSize: 0
        UsedReserveSize: 0
      }
    }
    PQPartitionsInside: 1
    PQPartitionsLimit: 1000000
    SchemeLimits {
      MaxDepth: 32
      MaxPaths: 10000
      MaxChildrenInDir: 100000
      MaxAclBytesSize: 10240
      MaxTableColumns: 200
      MaxTableColumnNameLength: 255
      MaxTableKeyColumns: 30
      MaxTableIndices: 20
      MaxShards: 200000
      MaxShardsInPath: 35000
      MaxConsistentCopyTargets: 10000
      MaxPathElementLength: 255
      ExtraPathSymbolsAllowed: "!\"#$%&\'()*+,-.:;<=>?@[\\]^_`{|}~"
      MaxPQPartitions: 1000000
      MaxTableCdcStreams: 5
      MaxExports: 10
      MaxImports: 10
      MaxColumnTableColumns: 10000
    }
  }
}
Path: "/Root/PQ/rt3.dc1--topic1"
 name rt3.dc1--topic1 version1
CallPersQueueGRPC request to localhost:2670
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--topic1"
  }
}
2025-11-28T18:22:31.445475Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
MetaResponse {
  CmdGetTopicMetadataResult {
    TopicInfo {
      Topic: "rt3.dc1--topic1"
      NumPartitions: 1
      Config {
        PartitionConfig {
          LifetimeSeconds: 86400
          LowWatermark: 8388608
          SourceIdLifetimeSeconds: 86400
          WriteSpeedInBytesPerSecond: 20000000
          BurstSize: 20000000
          SourceIdMaxCounts: 6000000
        }
        Version: 1
        LocalDC: true
        Codecs {
          Ids: 0
          Ids: 1
          Ids: 2
          Codecs: "raw"
          Codecs: "gzip"
          Codecs: "lzop"
        }
        TopicPath: "/Root/PQ/rt3.dc1--topic1"
        YdbDatabasePath: "/Root"
        Consumers {
          Name: "user"
          ReadFromTimestampsMs: 0
          FormatVersion: 0
          Version: 0
        }
      }
      ErrorCode: OK
    }
  }
}
=== Topic created, have version: 1
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.447223Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.447731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558104:3371], now have 1 active actors on pipe
2025-11-28T18:22:31.447834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.447841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.447997Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|be9ed2d7-6e9a2585-94ed8f1b-d9fd1696_0 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.448012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.448016Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.448021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.448024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.448028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.448037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.448039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.448042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.448057Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.448069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.448214Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|be9ed2d7-6e9a2585-94ed8f1b-d9fd1696_0"
    Status: Active
    SeqNo: 0
  }
}
2025-11-28T18:22:31.448434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558104:3371] destroyed
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|be9ed2d7-6e9a2585-94ed8f1b-d9fd1696_0"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 0
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.460808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558107:3372], now have 1 active actors on pipe
2025-11-28T18:22:31.460239Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.462541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.462559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.462793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 0 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.462865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 0 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.462897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 0 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.462906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 0 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.462925Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.462943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.462951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.462960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.462963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.462966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.462968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.462973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.463004Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 0
2025-11-28T18:22:31.463023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.463030Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 1
2025-11-28T18:22:31.463034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.463037Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 2
2025-11-28T18:22:31.463293Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 0 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1048790 count 1 nextOffset 1 batches 3
2025-11-28T18:22:31.463314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.463316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.463319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.463386Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 0,1 HeadOffset 0 endOffset 0 curOffset 1 d0000000000_00000000000000000000_00000_0000000001_00002? size 1048780 WTime 1764354151462
2025-11-28T18:22:31.464277Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.464300Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.467642Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 0 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.467669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.467686Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.467691Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.467698Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.467708Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 0, partNo: 0, Offset: 0 is stored on disk
2025-11-28T18:22:31.467712Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.467716Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 0, partNo: 1, Offset: 0 is stored on disk
2025-11-28T18:22:31.467721Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.467725Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 0, partNo: 2, Offset: 0 is stored on disk
2025-11-28T18:22:31.467759Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:930: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-11-28T18:22:31.467770Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:972: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-11-28T18:22:31.467774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.467779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.467781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.467783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.467786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.467794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.467809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.467850Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--topic1' partition 0 user user offset 0 partno 0 count 1 size 1024000 endOffset 1 max time lag 0ms effective offset 0
2025-11-28T18:22:31.467870Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 1048780 count 1 last offset 0, current partition end offset: 1
2025-11-28T18:22:31.467876Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-11-28T18:22:31.467886Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.467896Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 3. All 1 blobs are from cache.
2025-11-28T18:22:31.467915Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 1 blobs
2025-11-28T18:22:31.468065Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 1 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:31.468178Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.468200Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 1 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:31.468216Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 1 count 1 size 24704 from pos 0 cbcount 1
2025-11-28T18:22:31.468238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558107:3372] destroyed
2025-11-28T18:22:31.468245Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1022: Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp done, result 1764354151462 queuesize 0 startOffset 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 0
    Offset: 0
    PartNo: 0
    WriteTimestampMS: 1764354151462
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.469327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558113:3375], now have 1 active actors on pipe
2025-11-28T18:22:31.469431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.469443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.469470Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|8f1355b0-da40aa27-1ee322ac-18161624_1 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.469483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.469486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.469490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.469492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.469495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.469502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.469505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.469507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.469533Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.469548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|8f1355b0-da40aa27-1ee322ac-18161624_1"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2670
2025-11-28T18:22:31.469986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558113:3375] destroyed
2025-11-28T18:22:31.467902Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.468986Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.469660Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|8f1355b0-da40aa27-1ee322ac-18161624_1"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 1
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.479342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558116:3376], now have 1 active actors on pipe
2025-11-28T18:22:31.478899Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.480593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.480612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.480784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 1 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.481207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 1 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.481227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 1 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.481232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 1 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.481254Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.481273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.481281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.481289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.481295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.481299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.481301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.481306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.481331Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 0
2025-11-28T18:22:31.481347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.481354Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 1
2025-11-28T18:22:31.481358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.481362Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 2
2025-11-28T18:22:31.481952Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 1 partNo 2 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 1048790 count 1 nextOffset 2 batches 3
2025-11-28T18:22:31.481970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.481972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.481975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.482046Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 1,1 HeadOffset 1 endOffset 1 curOffset 2 d0000000000_00000000000000000001_00000_0000000001_00002? size 1048780 WTime 1764354151481
2025-11-28T18:22:31.482242Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.482263Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 1 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.485601Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 1 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.485622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.485643Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.485649Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.485657Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 1, partNo: 0, Offset: 1 is stored on disk
2025-11-28T18:22:31.485665Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.485669Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 1, partNo: 1, Offset: 1 is stored on disk
2025-11-28T18:22:31.485675Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.485684Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 1, partNo: 2, Offset: 1 is stored on disk
2025-11-28T18:22:31.485716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.485723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.485725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.485728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.485730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.485736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.485794Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.485829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.485925Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.486431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558116:3376] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 1
    Offset: 1
    PartNo: 0
    WriteTimestampMS: 1764354151481
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.487261Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.487649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558121:3379], now have 1 active actors on pipe
2025-11-28T18:22:31.487770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.487781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.487809Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|d1e5badd-fccc0f72-99201206-f4a3ca95_2 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.487823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.487830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.487835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.487837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.487841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.487850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.487858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.487861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.487873Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.487888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.487972Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.488222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558121:3379] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|d1e5badd-fccc0f72-99201206-f4a3ca95_2"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|d1e5badd-fccc0f72-99201206-f4a3ca95_2"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 2
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.495707Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.496937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558124:3380], now have 1 active actors on pipe
2025-11-28T18:22:31.497532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.497556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.497640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 2 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.498089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 2 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.498108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 2 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.498118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 2 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.498141Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.498164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.498172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.498182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.498188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.498192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.498194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.498200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.498235Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 0
2025-11-28T18:22:31.498254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.498261Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 1
2025-11-28T18:22:31.498265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.498268Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 2
2025-11-28T18:22:31.498525Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 2 partNo 2 FormedBlobsCount 0 NewHead: Offset 2 PartNo 0 PackedSize 1048790 count 1 nextOffset 3 batches 3
2025-11-28T18:22:31.498546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.498549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.498552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.498625Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 2,1 HeadOffset 2 endOffset 2 curOffset 3 d0000000000_00000000000000000002_00000_0000000001_00002? size 1048780 WTime 1764354151498
2025-11-28T18:22:31.499198Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.499212Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 2 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.502130Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 2 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.502154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.502175Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.502176Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.502182Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.502189Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 2, partNo: 0, Offset: 2 is stored on disk
2025-11-28T18:22:31.502193Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.502198Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 2, partNo: 1, Offset: 2 is stored on disk
2025-11-28T18:22:31.502202Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.502207Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 2, partNo: 2, Offset: 2 is stored on disk
2025-11-28T18:22:31.502234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.502243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.502245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.502247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.502249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.502256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.502270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.502361Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 2
    Offset: 2
    PartNo: 0
    WriteTimestampMS: 1764354151498
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:31.502815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558124:3380] destroyed
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.503421Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.503807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558129:3383], now have 1 active actors on pipe
2025-11-28T18:22:31.503925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.503935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.503961Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|5634f75d-36868768-a0e66cf2-6a63b103_3 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.503972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.503976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.503980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.503986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.503989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.503996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.503998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.504000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.504010Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.504036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.504098Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.504360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558129:3383] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|5634f75d-36868768-a0e66cf2-6a63b103_3"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|5634f75d-36868768-a0e66cf2-6a63b103_3"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 3
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.510981Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.511440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558132:3384], now have 1 active actors on pipe
2025-11-28T18:22:31.513150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.513175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.513255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 3 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.513334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 3 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.513352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 3 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.513363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 3 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.513386Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.513410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.513417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.513428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.513435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.513439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.513440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.513446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.513474Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 0
2025-11-28T18:22:31.513493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.513501Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 1
2025-11-28T18:22:31.513504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.513507Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 2
2025-11-28T18:22:31.513736Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 3 partNo 2 FormedBlobsCount 0 NewHead: Offset 3 PartNo 0 PackedSize 1048790 count 1 nextOffset 4 batches 3
2025-11-28T18:22:31.513760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.513762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.513766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.513847Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 3,1 HeadOffset 3 endOffset 3 curOffset 4 d0000000000_00000000000000000003_00000_0000000001_00002? size 1048780 WTime 1764354151513
2025-11-28T18:22:31.514041Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.514063Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 3 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.517295Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 3 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.517323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.517347Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.517348Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.517355Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.517366Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 3, partNo: 0, Offset: 3 is stored on disk
2025-11-28T18:22:31.517370Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.517374Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 3, partNo: 1, Offset: 3 is stored on disk
2025-11-28T18:22:31.517380Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.517385Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 3, partNo: 2, Offset: 3 is stored on disk
2025-11-28T18:22:31.517425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.517433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.517436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.517438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.517441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.517450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.517467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.517556Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.517903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558132:3384] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 3
    Offset: 3
    PartNo: 0
    WriteTimestampMS: 1764354151513
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.518575Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.519020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558137:3387], now have 1 active actors on pipe
2025-11-28T18:22:31.519136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.519147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.519179Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a940e7d4-e5bcdad6-84c054b-eb61ffa6_4 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.519192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.519199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.519205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.519206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.519210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.519229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.519236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.519239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.519253Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.519269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.519329Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a940e7d4-e5bcdad6-84c054b-eb61ffa6_4"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2670
2025-11-28T18:22:31.519588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558137:3387] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a940e7d4-e5bcdad6-84c054b-eb61ffa6_4"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 4
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.526188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.526201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.526206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.526210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.526212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.527065Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.527642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558140:3388], now have 1 active actors on pipe
2025-11-28T18:22:31.528690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.528708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.528792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 4 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.528857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 4 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.528867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 4 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.528875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 4 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.528894Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.528910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.528913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.528920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.528924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.528927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.528930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.528934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.528964Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 0
2025-11-28T18:22:31.528979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.528984Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 1
2025-11-28T18:22:31.528988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.528991Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 2
2025-11-28T18:22:31.529618Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 4 partNo 2 FormedBlobsCount 0 NewHead: Offset 4 PartNo 0 PackedSize 1048790 count 1 nextOffset 5 batches 3
2025-11-28T18:22:31.529632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.529634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.529638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.529708Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 4,1 HeadOffset 4 endOffset 4 curOffset 5 d0000000000_00000000000000000004_00000_0000000001_00002? size 1048780 WTime 1764354151529
2025-11-28T18:22:31.529886Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.529898Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 4 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.533496Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 4 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.533524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.533546Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.533545Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.533563Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.533572Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 4, partNo: 0, Offset: 4 is stored on disk
2025-11-28T18:22:31.533575Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.533580Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 4, partNo: 1, Offset: 4 is stored on disk
2025-11-28T18:22:31.533585Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.533590Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 4, partNo: 2, Offset: 4 is stored on disk
2025-11-28T18:22:31.533620Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.533632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.533634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.533637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.533639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.533646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.533660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.533757Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.534108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558140:3388] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 4
    Offset: 4
    PartNo: 0
    WriteTimestampMS: 1764354151528
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.534786Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.535174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558145:3391], now have 1 active actors on pipe
2025-11-28T18:22:31.535265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.535273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.535298Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|3bfcd41c-9a53c622-2fec466-ea53911f_5 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.535306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.535309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.535313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.535315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.535318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.535326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.535328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.535333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.535354Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.535363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.535400Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|3bfcd41c-9a53c622-2fec466-ea53911f_5"
    Status: Active
    SeqNo: 0
  }
}
2025-11-28T18:22:31.535583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558145:3391] destroyed
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|3bfcd41c-9a53c622-2fec466-ea53911f_5"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 5
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.543057Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.543493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558148:3392], now have 1 active actors on pipe
2025-11-28T18:22:31.544647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.544665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.544746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 5 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.544803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 5 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.544813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 5 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.544818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 5 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.544837Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.544859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.544867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.544875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.544882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.544886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.544888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.544893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.544924Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 0
2025-11-28T18:22:31.544942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.544945Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 1
2025-11-28T18:22:31.544950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.544953Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 2
2025-11-28T18:22:31.545173Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 5 partNo 2 FormedBlobsCount 0 NewHead: Offset 5 PartNo 0 PackedSize 1048790 count 1 nextOffset 6 batches 3
2025-11-28T18:22:31.545194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.545197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.545199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.545269Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 5,1 HeadOffset 5 endOffset 5 curOffset 6 d0000000000_00000000000000000005_00000_0000000001_00002? size 1048780 WTime 1764354151545
2025-11-28T18:22:31.546160Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.546182Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 5 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.549057Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 5 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.549084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.549097Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.549107Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.549113Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.549122Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 5, partNo: 0, Offset: 5 is stored on disk
2025-11-28T18:22:31.549130Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.549135Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 5, partNo: 1, Offset: 5 is stored on disk
2025-11-28T18:22:31.549140Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.549145Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 5, partNo: 2, Offset: 5 is stored on disk
2025-11-28T18:22:31.549176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.549183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.549185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.549187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.549189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.549196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.549212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.549293Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.549623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558148:3392] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 5
    Offset: 5
    PartNo: 0
    WriteTimestampMS: 1764354151544
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.550333Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.550701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558153:3395], now have 1 active actors on pipe
2025-11-28T18:22:31.550779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.550790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.550823Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|6a7b2c64-58888011-37af2270-29c2409a_6 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.550836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.550840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.550845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.550847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.550851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.550860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.550864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.550873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.550883Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.550899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.550945Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.551155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558153:3395] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|6a7b2c64-58888011-37af2270-29c2409a_6"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|6a7b2c64-58888011-37af2270-29c2409a_6"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 6
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.555034Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.555417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558156:3396], now have 1 active actors on pipe
2025-11-28T18:22:31.556544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.556561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.556634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 6 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.556706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 6 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.556732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 6 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.556744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 6 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.556768Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.556789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.556796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.556805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.556812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.556816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.556819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.556824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.556855Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 0
2025-11-28T18:22:31.556872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.556879Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 1
2025-11-28T18:22:31.556883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.556885Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 2
2025-11-28T18:22:31.557457Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 6 partNo 2 FormedBlobsCount 0 NewHead: Offset 6 PartNo 0 PackedSize 1048790 count 1 nextOffset 7 batches 3
2025-11-28T18:22:31.557478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.557481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.557484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.557553Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 6,1 HeadOffset 6 endOffset 6 curOffset 7 d0000000000_00000000000000000006_00000_0000000001_00002? size 1048780 WTime 1764354151557
2025-11-28T18:22:31.557748Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.557770Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 6 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.560475Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 6 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.560498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.560512Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.560519Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.560524Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.560531Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 6, partNo: 0, Offset: 6 is stored on disk
2025-11-28T18:22:31.560535Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.560539Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 6, partNo: 1, Offset: 6 is stored on disk
2025-11-28T18:22:31.560544Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.560552Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 6, partNo: 2, Offset: 6 is stored on disk
2025-11-28T18:22:31.560580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.560588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.560590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.560592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.560594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.560601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.560614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.560679Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.560935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558156:3396] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 6
    Offset: 6
    PartNo: 0
    WriteTimestampMS: 1764354151556
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.561461Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.561804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558161:3399], now have 1 active actors on pipe
2025-11-28T18:22:31.561867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.561876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.561906Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|3c5bead9-ae2b60ae-80b39107-6cfbbaa6_7 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.561917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.561924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.561928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.561931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.561934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.561944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.561951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.561954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.561964Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.561974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.562022Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.562200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558161:3399] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|3c5bead9-ae2b60ae-80b39107-6cfbbaa6_7"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|3c5bead9-ae2b60ae-80b39107-6cfbbaa6_7"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 7
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.567473Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.567913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558165:3401], now have 1 active actors on pipe
2025-11-28T18:22:31.569025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.569042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.569116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 7 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.569196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 7 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.569211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 7 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.569232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 7 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.569259Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.569279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.569287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.569296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.569303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.569308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.569311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.569316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.569346Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 0
2025-11-28T18:22:31.569362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.569369Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 1
2025-11-28T18:22:31.569373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.569377Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 2
2025-11-28T18:22:31.570024Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 7 partNo 2 FormedBlobsCount 0 NewHead: Offset 7 PartNo 0 PackedSize 1048790 count 1 nextOffset 8 batches 3
2025-11-28T18:22:31.570048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.570051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.570055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.570128Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 7,1 HeadOffset 7 endOffset 7 curOffset 8 d0000000000_00000000000000000007_00000_0000000001_00002? size 1048780 WTime 1764354151569
2025-11-28T18:22:31.570327Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.570350Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 7 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.573243Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 7 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.573270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.573289Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.573295Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.573301Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.573310Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 7, partNo: 0, Offset: 7 is stored on disk
2025-11-28T18:22:31.573314Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.573318Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 7, partNo: 1, Offset: 7 is stored on disk
2025-11-28T18:22:31.573327Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.573331Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 7, partNo: 2, Offset: 7 is stored on disk
2025-11-28T18:22:31.573368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.573376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.573378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.573380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.573382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.573392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000000_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.573395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000001_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.573398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000002_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.573400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000003_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.573405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000004_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.573408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000005_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.573411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000006_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.573413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000007_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.573416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 8 keys were taken away. Let's read 8390240 bytes
2025-11-28T18:22:31.573434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.573451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 8 blobs
2025-11-28T18:22:31.573457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000000_00000_0000000001_00002?
2025-11-28T18:22:31.573467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000001_00000_0000000001_00002?
2025-11-28T18:22:31.573480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000002_00000_0000000001_00002?
2025-11-28T18:22:31.573497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000003_00000_0000000001_00002?
2025-11-28T18:22:31.573507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000004_00000_0000000001_00002?
2025-11-28T18:22:31.573533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000005_00000_0000000001_00002?
2025-11-28T18:22:31.573545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000006_00000_0000000001_00002?
2025-11-28T18:22:31.573553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000007_00000_0000000001_00002?
2025-11-28T18:22:31.573561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 8 blobs for compaction
2025-11-28T18:22:31.573570Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 1 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.573577Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 1 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.573581Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 2 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.573585Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 3 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.573589Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 4 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.573596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 5 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.573600Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 6 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.573603Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 7 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.573614Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 8 blobs are from cache.
2025-11-28T18:22:31.573626Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.573636Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.573639Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.573643Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.573644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-28T18:22:31.573646Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.573649Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.573651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/8] d0000000000_00000000000000000000_00000_0000000001_00002?
2025-11-28T18:22:31.573652Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.573653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.573655Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.573656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000000_00000_0000000001_00002?, parameters.CurOffset=0
2025-11-28T18:22:31.573514Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.573797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 0.0/3
2025-11-28T18:22:31.573813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 0
2025-11-28T18:22:31.573827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 0:0 appended
CallPersQueueGRPC response:
2025-11-28T18:22:31.573963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 0.1/3
2025-11-28T18:22:31.573977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 1
2025-11-28T18:22:31.573980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 0:1 appended
2025-11-28T18:22:31.573995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 0.2/3
2025-11-28T18:22:31.574004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 2
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 7
    Offset: 7
    PartNo: 0
    WriteTimestampMS: 1764354151569
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.574219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 0 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1048790 count 1 nextOffset 1 batches 3
2025-11-28T18:22:31.574232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 0:2 appended
2025-11-28T18:22:31.574236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/8] d0000000000_00000000000000000001_00000_0000000001_00002?
2025-11-28T18:22:31.574239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.574242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000001_00000_0000000001_00002?, parameters.CurOffset=1
2025-11-28T18:22:31.574373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 1.0/3
2025-11-28T18:22:31.574387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 0
2025-11-28T18:22:31.574390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 1:0 appended
2025-11-28T18:22:31.574546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 1.1/3
2025-11-28T18:22:31.574558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 1
2025-11-28T18:22:31.574561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 1:1 appended
2025-11-28T18:22:31.574579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 1.2/3
2025-11-28T18:22:31.574586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 2
2025-11-28T18:22:31.574695Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.574781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 1 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2097545 count 2 nextOffset 2 batches 5
2025-11-28T18:22:31.574797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 1:2 appended
2025-11-28T18:22:31.574801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/8] d0000000000_00000000000000000002_00000_0000000001_00002?
2025-11-28T18:22:31.574803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.574806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000002_00000_0000000001_00002?, parameters.CurOffset=2
2025-11-28T18:22:31.574958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 2.0/3
2025-11-28T18:22:31.574973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 0
2025-11-28T18:22:31.574981Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 2:0 appended
2025-11-28T18:22:31.575469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 2.1/3
2025-11-28T18:22:31.575484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 1
2025-11-28T18:22:31.575487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 2:1 appended
2025-11-28T18:22:31.575503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 2.2/3
2025-11-28T18:22:31.575510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 2
2025-11-28T18:22:31.576088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 2 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3146300 count 3 nextOffset 3 batches 7
2025-11-28T18:22:31.576106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 2:2 appended
2025-11-28T18:22:31.576110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/8] d0000000000_00000000000000000003_00000_0000000001_00002?
2025-11-28T18:22:31.576112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.576115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000003_00000_0000000001_00002?, parameters.CurOffset=3
2025-11-28T18:22:31.576256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 3.0/3
2025-11-28T18:22:31.576269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 0
2025-11-28T18:22:31.576272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 3:0 appended
2025-11-28T18:22:31.576410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 3.1/3
2025-11-28T18:22:31.576429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 1
2025-11-28T18:22:31.576432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 3:1 appended
2025-11-28T18:22:31.576446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 3.2/3
2025-11-28T18:22:31.576449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 2
2025-11-28T18:22:31.577035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 3 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4195055 count 4 nextOffset 4 batches 9
2025-11-28T18:22:31.577053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 3:2 appended
2025-11-28T18:22:31.577058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/8] d0000000000_00000000000000000004_00000_0000000001_00002?
2025-11-28T18:22:31.577060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.577064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000004_00000_0000000001_00002?, parameters.CurOffset=4
2025-11-28T18:22:31.577199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 4.0/3
2025-11-28T18:22:31.577211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 0
2025-11-28T18:22:31.577214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 4:0 appended
2025-11-28T18:22:31.577537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 4.1/3
2025-11-28T18:22:31.577550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 1
2025-11-28T18:22:31.577553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 4:1 appended
2025-11-28T18:22:31.577568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 4.2/3
2025-11-28T18:22:31.577570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 2
2025-11-28T18:22:31.578135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 4 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 5243810 count 5 nextOffset 5 batches 11
2025-11-28T18:22:31.578152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 4:2 appended
2025-11-28T18:22:31.578157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/8] d0000000000_00000000000000000005_00000_0000000001_00002?
2025-11-28T18:22:31.578159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.578163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000005_00000_0000000001_00002?, parameters.CurOffset=5
2025-11-28T18:22:31.578297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 5.0/3
2025-11-28T18:22:31.578311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 0
2025-11-28T18:22:31.578318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 5:0 appended
2025-11-28T18:22:31.578838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 5.1/3
2025-11-28T18:22:31.578855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 1
2025-11-28T18:22:31.578860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 5:1 appended
2025-11-28T18:22:31.578875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 5.2/3
2025-11-28T18:22:31.578878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 2
2025-11-28T18:22:31.579845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 5 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 6292565 count 6 nextOffset 6 batches 13
2025-11-28T18:22:31.579860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 5:2 appended
2025-11-28T18:22:31.579864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/8] d0000000000_00000000000000000006_00000_0000000001_00002?
2025-11-28T18:22:31.579866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.579870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000006_00000_0000000001_00002?, parameters.CurOffset=6
2025-11-28T18:22:31.580001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 6.0/3
2025-11-28T18:22:31.580014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 0
2025-11-28T18:22:31.580017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 6:0 appended
2025-11-28T18:22:31.580147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 6.1/3
2025-11-28T18:22:31.580158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 1
2025-11-28T18:22:31.580160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 6:1 appended
2025-11-28T18:22:31.580172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 6.2/3
2025-11-28T18:22:31.580175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 2
2025-11-28T18:22:31.580718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 6 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 7341320 count 7 nextOffset 7 batches 15
2025-11-28T18:22:31.580749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 6:2 appended
2025-11-28T18:22:31.580754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/8] d0000000000_00000000000000000007_00000_0000000001_00002?
2025-11-28T18:22:31.580755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.580759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000007_00000_0000000001_00002?, parameters.CurOffset=7
2025-11-28T18:22:31.580887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 7.0/3
2025-11-28T18:22:31.580901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 0
2025-11-28T18:22:31.580903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 7:0 appended
2025-11-28T18:22:31.581220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 7.1/3
2025-11-28T18:22:31.581235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 1
2025-11-28T18:22:31.581238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 7:1 appended
2025-11-28T18:22:31.581253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 7.2/3
2025-11-28T18:22:31.581257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 2
2025-11-28T18:22:31.588170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 7 partNo 2 result is x0000000000_00000000000000000000_00000_0000000007_00016 size 8365339
2025-11-28T18:22:31.588199Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1110: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000000_00000_0000000007_00016 new key d0000000000_00000000000000000000_00000_0000000007_00016 size 8365339 WTime 1764354151588
2025-11-28T18:22:31.588244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 7 partNo 2 FormedBlobsCount 1 NewHead: Offset 7 PartNo 2 PackedSize 24736 count 1 nextOffset 8 batches 1
2025-11-28T18:22:31.588253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 7:2 appended
2025-11-28T18:22:31.588314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 7,1 HeadOffset 0 endOffset 0 curOffset 8 d0000000000_00000000000000000007_00002_0000000001_00000| size 24726 WTime 1764354151588
2025-11-28T18:22:31.588376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558165:3401] destroyed
2025-11-28T18:22:31.588390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558170:3404], now have 1 active actors on pipe
2025-11-28T18:22:31.588410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.588416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.588431Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.588447Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 7 size 8365339
2025-11-28T18:22:31.588458Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 7 partNo 2 count 1 size 24726
2025-11-28T18:22:31.588498Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|1e33758b-4b32deda-9c2f6aea-8d626da3_8 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.588512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.588515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.588520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.588522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.588526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.588537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.588540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.588542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.588557Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.588608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.588711Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.589038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558170:3404] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|1e33758b-4b32deda-9c2f6aea-8d626da3_8"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|1e33758b-4b32deda-9c2f6aea-8d626da3_8"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 8
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.595930Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.596461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558176:3407], now have 1 active actors on pipe
2025-11-28T18:22:31.597600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.597618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.597691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 8 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.597763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 8 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.597778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 8 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.597784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 8 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.597819Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 0 count 7 size 8365339 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.597830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 7 count 1 size 24726 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.597855Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.597878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.597884Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 7 parts 16 suffix '0' size 8365339
2025-11-28T18:22:31.597886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.597890Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 7 partno 2 count 1 parts 0 suffix '124' size 24726
2025-11-28T18:22:31.597898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.597900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.597905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.597907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.597911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.597944Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 0
2025-11-28T18:22:31.597963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.597970Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 1
2025-11-28T18:22:31.597974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.597977Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 2
2025-11-28T18:22:31.598219Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 8 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 1048790 count 1 nextOffset 9 batches 3
2025-11-28T18:22:31.598240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.598243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.598249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.598330Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 8,1 HeadOffset 8 endOffset 8 curOffset 9 d0000000000_00000000000000000008_00000_0000000001_00002? size 1048780 WTime 1764354151597
2025-11-28T18:22:31.598555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.598598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-28T18:22:31.598606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.598614Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.598631Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 8 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.601355Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 8 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.601380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.601403Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.601411Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.601415Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.601424Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 8, partNo: 0, Offset: 8 is stored on disk
2025-11-28T18:22:31.601428Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.601433Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 8, partNo: 1, Offset: 8 is stored on disk
2025-11-28T18:22:31.601443Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.601447Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 8, partNo: 2, Offset: 8 is stored on disk
2025-11-28T18:22:31.601479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.601486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.601488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.601491Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.601493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.601545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.601567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.601590Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.601601Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000000_00000_0000000001_00002?(+) to d0000000000_00000000000000000000_00000_0000000001_00002?(+)
2025-11-28T18:22:31.601608Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000001_00000_0000000001_00002?(+) to d0000000000_00000000000000000001_00000_0000000001_00002?(+)
2025-11-28T18:22:31.601610Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000002_00000_0000000001_00002?(+) to d0000000000_00000000000000000002_00000_0000000001_00002?(+)
2025-11-28T18:22:31.601612Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000003_00000_0000000001_00002?(+) to d0000000000_00000000000000000003_00000_0000000001_00002?(+)
2025-11-28T18:22:31.601615Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000004_00000_0000000001_00002?(+) to d0000000000_00000000000000000004_00000_0000000001_00002?(+)
2025-11-28T18:22:31.601617Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000005_00000_0000000001_00002?(+) to d0000000000_00000000000000000005_00000_0000000001_00002?(+)
2025-11-28T18:22:31.601619Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000006_00000_0000000001_00002?(+) to d0000000000_00000000000000000006_00000_0000000001_00002?(+)
2025-11-28T18:22:31.601622Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000007_00000_0000000001_00002?(+) to d0000000000_00000000000000000007_00000_0000000001_00002?(+)
2025-11-28T18:22:31.601666Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.601948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558176:3407] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 8
    Offset: 8
    PartNo: 0
    WriteTimestampMS: 1764354151597
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.602351Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 0 count 1 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.602364Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 1 count 1 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.602370Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 2 count 1 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.602374Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 3 count 1 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.602379Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 4 count 1 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.602383Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 5 count 1 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.602386Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 6 count 1 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.602390Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 7 count 1 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.602402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.602413Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.602418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.602421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.602423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.602425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.602426Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.602427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.602433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.602441Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.602446Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.602450Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.602455Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.602476Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.602480Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.602485Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.602651Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.602993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558184:3412], now have 1 active actors on pipe
2025-11-28T18:22:31.603011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.603021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.603051Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|55da6d92-dfc4b77d-9e01917e-48fb586_9 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.603063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.603067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.603072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.603078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.603081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.603089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.603098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.603101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.603110Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.603125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.603169Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.603362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558184:3412] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|55da6d92-dfc4b77d-9e01917e-48fb586_9"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|55da6d92-dfc4b77d-9e01917e-48fb586_9"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 9
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.609549Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.610048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558188:3415], now have 1 active actors on pipe
2025-11-28T18:22:31.610994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.611010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.611192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 9 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.611601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 9 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.611616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 9 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.611621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 9 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.611643Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.611667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.611673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.611684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.611687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.611691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.611694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.611699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.611733Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 0
2025-11-28T18:22:31.611755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.611758Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 1
2025-11-28T18:22:31.611761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.611764Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 2
2025-11-28T18:22:31.612324Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 9 partNo 2 FormedBlobsCount 0 NewHead: Offset 9 PartNo 0 PackedSize 1048790 count 1 nextOffset 10 batches 3
2025-11-28T18:22:31.612344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.612347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.612351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.612423Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 9,1 HeadOffset 9 endOffset 9 curOffset 10 d0000000000_00000000000000000009_00000_0000000001_00002? size 1048780 WTime 1764354151612
2025-11-28T18:22:31.612628Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.612645Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 9 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.615577Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 9 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.615603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.615623Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.615627Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.615634Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.615644Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 9, partNo: 0, Offset: 9 is stored on disk
2025-11-28T18:22:31.615652Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.615657Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 9, partNo: 1, Offset: 9 is stored on disk
2025-11-28T18:22:31.615663Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.615672Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 9, partNo: 2, Offset: 9 is stored on disk
2025-11-28T18:22:31.615708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.615715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.615717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.615719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.615721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.615731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.615752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.615847Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 9
    Offset: 9
    PartNo: 0
    WriteTimestampMS: 1764354151611
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:31.616145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558188:3415] destroyed
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.616764Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.617177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558196:3418], now have 1 active actors on pipe
2025-11-28T18:22:31.617287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.617297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.617329Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|fc60b7c8-2981cf6b-f27879f5-108ef984_10 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.617343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.617347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.617351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.617353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.617357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.617367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.617375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.617377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.617391Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.617406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.617464Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.617672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558196:3418] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|fc60b7c8-2981cf6b-f27879f5-108ef984_10"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|fc60b7c8-2981cf6b-f27879f5-108ef984_10"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 10
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.624452Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.624930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558199:3419], now have 1 active actors on pipe
2025-11-28T18:22:31.626190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.626213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.626286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 10 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.626405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 10 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.626421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 10 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.626426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 10 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.626450Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.626488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.626496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.626505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.626512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.626517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.626519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.626524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.626556Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 0
2025-11-28T18:22:31.626573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.626580Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 1
2025-11-28T18:22:31.626584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.626586Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 2
2025-11-28T18:22:31.626801Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 10 partNo 2 FormedBlobsCount 0 NewHead: Offset 10 PartNo 0 PackedSize 1048790 count 1 nextOffset 11 batches 3
2025-11-28T18:22:31.626822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.626825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.626828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.626898Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 10,1 HeadOffset 10 endOffset 10 curOffset 11 d0000000000_00000000000000000010_00000_0000000001_00002? size 1048780 WTime 1764354151626
2025-11-28T18:22:31.627090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-28T18:22:31.627106Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.627118Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 10 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.630316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 10 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.630339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.630363Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.630369Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.630379Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 10, partNo: 0, Offset: 10 is stored on disk
2025-11-28T18:22:31.630383Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.630389Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 10, partNo: 1, Offset: 10 is stored on disk
2025-11-28T18:22:31.630394Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.630399Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 10, partNo: 2, Offset: 10 is stored on disk
2025-11-28T18:22:31.630431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.630439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.630441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.630443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.630447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.630455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.630485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.630541Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.630621Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 10
    Offset: 10
    PartNo: 0
    WriteTimestampMS: 1764354151626
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:31.630973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558199:3419] destroyed
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.631532Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.632011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558204:3422], now have 1 active actors on pipe
2025-11-28T18:22:31.632034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.632044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.632079Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a4becae9-c45ea81a-f7d6d52-6171e060_11 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.632091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.632095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.632100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.632107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.632110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.632118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.632120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.632123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.632135Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.632146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.632205Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a4becae9-c45ea81a-f7d6d52-6171e060_11"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2670
2025-11-28T18:22:31.633345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558204:3422] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a4becae9-c45ea81a-f7d6d52-6171e060_11"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 11
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.638920Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.639520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558207:3423], now have 1 active actors on pipe
2025-11-28T18:22:31.640591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.640611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.640686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 11 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.640755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 11 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.640771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 11 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.640776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 11 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.640798Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.640820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.640824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.640840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.640847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.640851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.640854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.640860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.640896Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 0
2025-11-28T18:22:31.640915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.640922Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 1
2025-11-28T18:22:31.640927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.640931Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 2
2025-11-28T18:22:31.641332Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 11 partNo 2 FormedBlobsCount 0 NewHead: Offset 11 PartNo 0 PackedSize 1048790 count 1 nextOffset 12 batches 3
2025-11-28T18:22:31.641351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.641355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.641358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.641432Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 11,1 HeadOffset 11 endOffset 11 curOffset 12 d0000000000_00000000000000000011_00000_0000000001_00002? size 1048780 WTime 1764354151641
2025-11-28T18:22:31.641624Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.641644Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 11 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.644545Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 11 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.644575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.644591Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.644600Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.644607Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.644621Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 11, partNo: 0, Offset: 11 is stored on disk
2025-11-28T18:22:31.644630Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.644637Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 11, partNo: 1, Offset: 11 is stored on disk
2025-11-28T18:22:31.644643Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.644647Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 11, partNo: 2, Offset: 11 is stored on disk
2025-11-28T18:22:31.644683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.644690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.644692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.644695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.644697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.644706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.644732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.644811Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.645133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558207:3423] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 11
    Offset: 11
    PartNo: 0
    WriteTimestampMS: 1764354151640
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.645749Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.646107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558212:3426], now have 1 active actors on pipe
2025-11-28T18:22:31.646187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.646199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.646230Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9db2e781-333df62e-40693d0e-c209f8d2_12 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.646246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.646249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.646255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.646260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.646264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.646274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.646280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.646283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.646295Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.646310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.646369Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.646593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558212:3426] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9db2e781-333df62e-40693d0e-c209f8d2_12"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9db2e781-333df62e-40693d0e-c209f8d2_12"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 12
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.652409Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.652896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558215:3427], now have 1 active actors on pipe
2025-11-28T18:22:31.653821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.653841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.653917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 12 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.653988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 12 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.654003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 12 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.654009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 12 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.654031Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.654052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.654060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.654069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.654076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.654080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.654086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.654091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.654126Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 0
2025-11-28T18:22:31.654143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.654150Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 1
2025-11-28T18:22:31.654154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.654157Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 2
2025-11-28T18:22:31.654374Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 12 partNo 2 FormedBlobsCount 0 NewHead: Offset 12 PartNo 0 PackedSize 1048790 count 1 nextOffset 13 batches 3
2025-11-28T18:22:31.654394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.654397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.654401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.654491Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 12,1 HeadOffset 12 endOffset 12 curOffset 13 d0000000000_00000000000000000012_00000_0000000001_00002? size 1048780 WTime 1764354151654
2025-11-28T18:22:31.654685Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.654704Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 12 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.657394Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 12 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.657418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.657433Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.657441Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.657452Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.657462Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 12, partNo: 0, Offset: 12 is stored on disk
2025-11-28T18:22:31.657469Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.657474Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 12, partNo: 1, Offset: 12 is stored on disk
2025-11-28T18:22:31.657479Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.657484Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 12, partNo: 2, Offset: 12 is stored on disk
2025-11-28T18:22:31.657515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.657522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.657524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.657526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.657528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.657536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.657559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.657650Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 12
    Offset: 12
    PartNo: 0
    WriteTimestampMS: 1764354151653
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:31.657989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558215:3427] destroyed
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.659035Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.659464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558224:3432], now have 1 active actors on pipe
2025-11-28T18:22:31.659542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.659552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.659586Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|d209e866-e214a317-ac6df9f1-9bbbad14_13 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.659599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.659607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.659613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.659619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.659625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.659634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.659640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.659643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.659655Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.659672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.659756Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|d209e866-e214a317-ac6df9f1-9bbbad14_13"
    Status: Active
    SeqNo: 0
  }
}
2025-11-28T18:22:31.660002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558224:3432] destroyed
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|d209e866-e214a317-ac6df9f1-9bbbad14_13"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 13
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.666308Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.666772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558227:3433], now have 1 active actors on pipe
2025-11-28T18:22:31.667875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.667891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.667976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 13 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.668097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 13 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.668112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 13 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.668116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 13 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.668132Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.668153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.668160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.668167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.668169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.668171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.668173Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.668176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.668199Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 0
2025-11-28T18:22:31.668215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.668221Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 1
2025-11-28T18:22:31.668224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.668226Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 2
2025-11-28T18:22:31.668416Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 13 partNo 2 FormedBlobsCount 0 NewHead: Offset 13 PartNo 0 PackedSize 1048790 count 1 nextOffset 14 batches 3
2025-11-28T18:22:31.668428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.668430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.668431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.668474Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 13,1 HeadOffset 13 endOffset 13 curOffset 14 d0000000000_00000000000000000013_00000_0000000001_00002? size 1048780 WTime 1764354151668
2025-11-28T18:22:31.668787Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.668803Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 13 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.672294Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 13 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.672315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.672332Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.672339Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.672340Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.672346Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 13, partNo: 0, Offset: 13 is stored on disk
2025-11-28T18:22:31.672348Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.672350Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 13, partNo: 1, Offset: 13 is stored on disk
2025-11-28T18:22:31.672354Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.672356Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 13, partNo: 2, Offset: 13 is stored on disk
2025-11-28T18:22:31.672378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.672385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.672387Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.672388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.672389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.672393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.672404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.672473Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 13
    Offset: 13
    PartNo: 0
    WriteTimestampMS: 1764354151668
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:31.672805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558227:3433] destroyed
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.673442Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.674235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558232:3436], now have 1 active actors on pipe
2025-11-28T18:22:31.674335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.674345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.674378Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a199cff3-18f69a30-37b17faa-7cd890ee_14 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.674391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.674393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.674396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.674398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.674400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.674408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.674412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.674414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.674424Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.674431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.674487Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.674699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558232:3436] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a199cff3-18f69a30-37b17faa-7cd890ee_14"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a199cff3-18f69a30-37b17faa-7cd890ee_14"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 14
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.678415Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.678909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558235:3437], now have 1 active actors on pipe
2025-11-28T18:22:31.679937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.679952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.680019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 14 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.680070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 14 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.680084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 14 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.680088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 14 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.680106Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.680126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.680132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.680138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.680139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.680142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.680143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.680146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.680161Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 0
2025-11-28T18:22:31.680170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.680177Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 1
2025-11-28T18:22:31.680179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.680181Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 2
2025-11-28T18:22:31.680369Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 14 partNo 2 FormedBlobsCount 0 NewHead: Offset 14 PartNo 0 PackedSize 1048790 count 1 nextOffset 15 batches 3
2025-11-28T18:22:31.680381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.680383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.680384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.680421Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 14,1 HeadOffset 14 endOffset 14 curOffset 15 d0000000000_00000000000000000014_00000_0000000001_00002? size 1048780 WTime 1764354151680
2025-11-28T18:22:31.680665Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.680677Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 14 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.683393Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 14 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.683414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.683429Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.683433Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.683438Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 14, partNo: 0, Offset: 14 is stored on disk
2025-11-28T18:22:31.683440Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.683442Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 14, partNo: 1, Offset: 14 is stored on disk
2025-11-28T18:22:31.683443Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.683445Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.683447Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 14, partNo: 2, Offset: 14 is stored on disk
2025-11-28T18:22:31.683466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.683467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.683468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.683469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.683471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.683475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.683482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.683556Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 14
    Offset: 14
    PartNo: 0
    WriteTimestampMS: 1764354151679
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:31.683884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558235:3437] destroyed
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.684489Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.684893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558240:3440], now have 1 active actors on pipe
2025-11-28T18:22:31.685052Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.684937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.684947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.684970Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|116f620-1c91079d-3dfd580-438dc0c5_15 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.684976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.684977Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.684980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.684982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.684984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.684989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.684991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.684992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.685001Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.685007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|116f620-1c91079d-3dfd580-438dc0c5_15"
    Status: Active
    SeqNo: 0
  }
}
2025-11-28T18:22:31.685272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558240:3440] destroyed
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|116f620-1c91079d-3dfd580-438dc0c5_15"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 15
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.689493Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.689993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558243:3441], now have 1 active actors on pipe
2025-11-28T18:22:31.691261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.691277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.691357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 15 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.691437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 15 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.691454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 15 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.691458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 15 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.691480Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.691494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.691501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.691507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.691511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.691514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.691516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.691521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.691549Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 0
2025-11-28T18:22:31.691568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.691570Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 1
2025-11-28T18:22:31.691574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.691577Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 2
2025-11-28T18:22:31.691794Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 15 partNo 2 FormedBlobsCount 0 NewHead: Offset 15 PartNo 0 PackedSize 1048790 count 1 nextOffset 16 batches 3
2025-11-28T18:22:31.691806Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.691810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.691813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.691875Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 15,1 HeadOffset 15 endOffset 15 curOffset 16 d0000000000_00000000000000000015_00000_0000000001_00002? size 1048780 WTime 1764354151691
2025-11-28T18:22:31.692112Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.692126Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 15 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.695652Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 15
    Offset: 15
    PartNo: 0
    WriteTimestampMS: 1764354151691
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.696706Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.695434Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 15 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.695459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.695478Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.695483Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.695485Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.695494Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 15, partNo: 0, Offset: 15 is stored on disk
2025-11-28T18:22:31.695497Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.695499Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 15, partNo: 1, Offset: 15 is stored on disk
2025-11-28T18:22:31.695503Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.695505Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 15, partNo: 2, Offset: 15 is stored on disk
2025-11-28T18:22:31.695531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.695532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.695534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.695536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.695538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.695545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000008_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.695547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000009_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.695548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000010_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.695550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000011_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.695551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000012_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.695553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000013_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.695555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000014_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.695557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000015_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.695559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 8 keys were taken away. Let's read 8390240 bytes
2025-11-28T18:22:31.695571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.695581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 8 blobs
2025-11-28T18:22:31.695583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000008_00000_0000000001_00002?
2025-11-28T18:22:31.695589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000009_00000_0000000001_00002?
2025-11-28T18:22:31.695594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000010_00000_0000000001_00002?
2025-11-28T18:22:31.695601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000011_00000_0000000001_00002?
2025-11-28T18:22:31.695605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000012_00000_0000000001_00002?
2025-11-28T18:22:31.695614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000013_00000_0000000001_00002?
2025-11-28T18:22:31.695619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000014_00000_0000000001_00002?
2025-11-28T18:22:31.695623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000015_00000_0000000001_00002?
2025-11-28T18:22:31.695629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 8 blobs for compaction
2025-11-28T18:22:31.695640Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 8 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.695646Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 9 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.695651Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 10 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.695655Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 11 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.695657Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 12 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.695662Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 13 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.695664Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 14 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.695666Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 15 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.695676Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 8 blobs are from cache.
2025-11-28T18:22:31.695687Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.695691Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.695696Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.695699Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.695701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-28T18:22:31.695702Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.695706Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.695707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/8] d0000000000_00000000000000000008_00000_0000000001_00002?
2025-11-28T18:22:31.695709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.695710Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.695711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000008_00000_0000000001_00002?, parameters.CurOffset=8
2025-11-28T18:22:31.695714Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.695863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 8.0/3
2025-11-28T18:22:31.695871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 0
2025-11-28T18:22:31.695876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 8:0 appended
2025-11-28T18:22:31.696041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 8.1/3
2025-11-28T18:22:31.696045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 1
2025-11-28T18:22:31.696048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 8:1 appended
2025-11-28T18:22:31.696064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 8.2/3
2025-11-28T18:22:31.696066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 2
2025-11-28T18:22:31.696298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 8 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 1048790 count 1 nextOffset 9 batches 3
2025-11-28T18:22:31.696304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 8:2 appended
2025-11-28T18:22:31.696309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/8] d0000000000_00000000000000000009_00000_0000000001_00002?
2025-11-28T18:22:31.696311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.696314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000009_00000_0000000001_00002?, parameters.CurOffset=9
2025-11-28T18:22:31.696470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 9.0/3
2025-11-28T18:22:31.696474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 0
2025-11-28T18:22:31.696477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 9:0 appended
2025-11-28T18:22:31.696637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 9.1/3
2025-11-28T18:22:31.696641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 1
2025-11-28T18:22:31.696646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 9:1 appended
2025-11-28T18:22:31.696661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 9.2/3
2025-11-28T18:22:31.696666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 2
2025-11-28T18:22:31.696869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 9 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 2097545 count 2 nextOffset 10 batches 5
2025-11-28T18:22:31.696874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 9:2 appended
2025-11-28T18:22:31.696876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/8] d0000000000_00000000000000000010_00000_0000000001_00002?
2025-11-28T18:22:31.696877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.696879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000010_00000_0000000001_00002?, parameters.CurOffset=10
2025-11-28T18:22:31.697016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 10.0/3
2025-11-28T18:22:31.697018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 0
2025-11-28T18:22:31.697020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 10:0 appended
2025-11-28T18:22:31.697163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 10.1/3
2025-11-28T18:22:31.697166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 1
2025-11-28T18:22:31.697169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 10:1 appended
2025-11-28T18:22:31.697181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 10.2/3
2025-11-28T18:22:31.697184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 2
2025-11-28T18:22:31.697352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 10 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 3146300 count 3 nextOffset 11 batches 7
2025-11-28T18:22:31.697355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 10:2 appended
2025-11-28T18:22:31.697357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/8] d0000000000_00000000000000000011_00000_0000000001_00002?
2025-11-28T18:22:31.697358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.697360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000011_00000_0000000001_00002?, parameters.CurOffset=11
2025-11-28T18:22:31.697499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 11.0/3
2025-11-28T18:22:31.697503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 0
2025-11-28T18:22:31.697506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 11:0 appended
2025-11-28T18:22:31.697826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 11.1/3
2025-11-28T18:22:31.697828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 1
2025-11-28T18:22:31.697830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 11:1 appended
2025-11-28T18:22:31.697840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 11.2/3
2025-11-28T18:22:31.697842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 2
2025-11-28T18:22:31.697995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 11 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 4195055 count 4 nextOffset 12 batches 9
2025-11-28T18:22:31.697998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 11:2 appended
2025-11-28T18:22:31.698000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/8] d0000000000_00000000000000000012_00000_0000000001_00002?
2025-11-28T18:22:31.698003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.698005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000012_00000_0000000001_00002?, parameters.CurOffset=12
2025-11-28T18:22:31.698140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 12.0/3
2025-11-28T18:22:31.698142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 0
2025-11-28T18:22:31.698143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 12:0 appended
2025-11-28T18:22:31.698283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 12.1/3
2025-11-28T18:22:31.698286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 1
2025-11-28T18:22:31.698288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 12:1 appended
2025-11-28T18:22:31.698300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 12.2/3
2025-11-28T18:22:31.698303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 2
2025-11-28T18:22:31.699012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 12 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 5243810 count 5 nextOffset 13 batches 11
2025-11-28T18:22:31.699026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 12:2 appended
2025-11-28T18:22:31.699031Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/8] d0000000000_00000000000000000013_00000_0000000001_00002?
2025-11-28T18:22:31.699032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.699035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000013_00000_0000000001_00002?, parameters.CurOffset=13
2025-11-28T18:22:31.699179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 13.0/3
2025-11-28T18:22:31.699186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 0
2025-11-28T18:22:31.699188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 13:0 appended
2025-11-28T18:22:31.699342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 13.1/3
2025-11-28T18:22:31.699351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 1
2025-11-28T18:22:31.699353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 13:1 appended
2025-11-28T18:22:31.699364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 13.2/3
2025-11-28T18:22:31.699368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 2
2025-11-28T18:22:31.699920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 13 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 6292565 count 6 nextOffset 14 batches 13
2025-11-28T18:22:31.699936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 13:2 appended
2025-11-28T18:22:31.699941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/8] d0000000000_00000000000000000014_00000_0000000001_00002?
2025-11-28T18:22:31.699942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.699945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000014_00000_0000000001_00002?, parameters.CurOffset=14
2025-11-28T18:22:31.700096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 14.0/3
2025-11-28T18:22:31.700101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 0
2025-11-28T18:22:31.700103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 14:0 appended
2025-11-28T18:22:31.700595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 14.1/3
2025-11-28T18:22:31.700602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 1
2025-11-28T18:22:31.700606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 14:1 appended
2025-11-28T18:22:31.700618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 14.2/3
2025-11-28T18:22:31.700620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 2
2025-11-28T18:22:31.701447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 14 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 7341320 count 7 nextOffset 15 batches 15
2025-11-28T18:22:31.701452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 14:2 appended
2025-11-28T18:22:31.701455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/8] d0000000000_00000000000000000015_00000_0000000001_00002?
2025-11-28T18:22:31.701456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.701457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000015_00000_0000000001_00002?, parameters.CurOffset=15
2025-11-28T18:22:31.701591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 15.0/3
2025-11-28T18:22:31.701596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 0
2025-11-28T18:22:31.701598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 15:0 appended
2025-11-28T18:22:31.701729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 15.1/3
2025-11-28T18:22:31.701731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 1
2025-11-28T18:22:31.706325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 15 partNo 1 result is x0000000000_00000000000000000007_00002_0000000008_00015 size 7878063
2025-11-28T18:22:31.706342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 15:1 appended
2025-11-28T18:22:31.706357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 15.2/3
2025-11-28T18:22:31.706360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 2
2025-11-28T18:22:31.706373Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1110: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000007_00002_0000000008_00015 new key d0000000000_00000000000000000007_00002_0000000008_00015 size 7878063 WTime 1764354151705
2025-11-28T18:22:31.706651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 15 partNo 2 FormedBlobsCount 1 NewHead: Offset 15 PartNo 1 PackedSize 536763 count 1 nextOffset 16 batches 2
2025-11-28T18:22:31.706665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 15:2 appended
2025-11-28T18:22:31.706715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 15,1 HeadOffset 7 endOffset 8 curOffset 16 d0000000000_00000000000000000015_00001_0000000001_00001| size 536753 WTime 1764354151705
2025-11-28T18:22:31.706843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558243:3441] destroyed
2025-11-28T18:22:31.706852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558248:3444], now have 1 active actors on pipe
2025-11-28T18:22:31.706871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.706878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.706893Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.706903Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 7 partNo 2 count 8 size 7878063
2025-11-28T18:22:31.706907Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 15 partNo 1 count 1 size 536753
2025-11-28T18:22:31.706939Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b4101dfe-d32c48c6-95fdd59d-4f30a9f4_16 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.706946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.706948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.706952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.706954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.706957Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.706965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.706966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.706968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.706979Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.707024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.707485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558248:3444] destroyed
2025-11-28T18:22:31.707153Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b4101dfe-d32c48c6-95fdd59d-4f30a9f4_16"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b4101dfe-d32c48c6-95fdd59d-4f30a9f4_16"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 16
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.715807Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.716407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558252:3446], now have 1 active actors on pipe
2025-11-28T18:22:31.718090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.718109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.718191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 16 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.718236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 16 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.718242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 16 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.718246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 16 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.718264Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.718278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.718281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.718288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.718290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.718293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.718295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.718298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.718321Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 0
2025-11-28T18:22:31.718333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.718335Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 1
2025-11-28T18:22:31.718337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.718339Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 2
2025-11-28T18:22:31.719393Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 16 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 1048790 count 1 nextOffset 17 batches 3
2025-11-28T18:22:31.719416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.719419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.719422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.719486Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 16,1 HeadOffset 16 endOffset 16 curOffset 17 d0000000000_00000000000000000016_00000_0000000001_00002? size 1048780 WTime 1764354151718
2025-11-28T18:22:31.719760Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.719772Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 16 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.723997Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 16 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.724011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 7 count 8 size 7878063 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.724013Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 15 count 1 size 536753 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.724019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.724037Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.724042Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.724049Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 16, partNo: 0, Offset: 16 is stored on disk
2025-11-28T18:22:31.724052Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.724054Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 16, partNo: 1, Offset: 16 is stored on disk
2025-11-28T18:22:31.724058Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.724063Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 16, partNo: 2, Offset: 16 is stored on disk
2025-11-28T18:22:31.724090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.724092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.724095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.724097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.724099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.724105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:178: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-11-28T18:22:31.724107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.724203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.724204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.724205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.724206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.724208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.724230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.724244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.724254Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.724259Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000008_00000_0000000001_00002?(+) to d0000000000_00000000000000000008_00000_0000000001_00002?(+)
2025-11-28T18:22:31.724260Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000009_00000_0000000001_00002?(+) to d0000000000_00000000000000000009_00000_0000000001_00002?(+)
2025-11-28T18:22:31.724261Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000010_00000_0000000001_00002?(+) to d0000000000_00000000000000000010_00000_0000000001_00002?(+)
2025-11-28T18:22:31.724262Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000011_00000_0000000001_00002?(+) to d0000000000_00000000000000000011_00000_0000000001_00002?(+)
2025-11-28T18:22:31.724263Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000012_00000_0000000001_00002?(+) to d0000000000_00000000000000000012_00000_0000000001_00002?(+)
2025-11-28T18:22:31.724264Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000013_00000_0000000001_00002?(+) to d0000000000_00000000000000000013_00000_0000000001_00002?(+)
2025-11-28T18:22:31.724266Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000014_00000_0000000001_00002?(+) to d0000000000_00000000000000000014_00000_0000000001_00002?(+)
2025-11-28T18:22:31.724268Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000015_00000_0000000001_00002?(+) to d0000000000_00000000000000000015_00000_0000000001_00002?(+)
2025-11-28T18:22:31.724269Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000007_00002_0000000001_00000|(+) to d0000000000_00000000000000000007_00002_0000000001_00000|(+)
2025-11-28T18:22:31.724598Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.724512Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.724522Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 7 partno 2 count 8 parts 15 suffix '0' size 7878063
2025-11-28T18:22:31.724524Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 15 partno 1 count 1 parts 1 suffix '124' size 536753
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 16
    Offset: 16
    PartNo: 0
    WriteTimestampMS: 1764354151718
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:31.724984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558252:3446] destroyed
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.725309Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 8 count 1 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.725316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 9 count 1 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.725321Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 10 count 1 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.725325Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 11 count 1 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.725331Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 12 count 1 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.725334Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 13 count 1 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.725337Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 14 count 1 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.725340Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 15 count 1 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.725345Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 7 count 1 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.725357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.725368Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.725376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.725378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.725380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.725382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.725384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.725390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.725621Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.725517Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.725529Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.725541Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.725545Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.725549Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.725556Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.725561Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.725566Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.725570Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 7 partno 2 count 1 parts 0 suffix '124' size 24726
2025-11-28T18:22:31.726013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558264:3452], now have 1 active actors on pipe
2025-11-28T18:22:31.726294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.726302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.726327Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9f3c110b-dffda47a-cbb2014-99ed3357_17 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.726334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.726337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.726341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.726344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.726346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.726354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.726357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.726360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.726371Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.726379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.726597Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.726859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558264:3452] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9f3c110b-dffda47a-cbb2014-99ed3357_17"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2670
2025-11-28T18:22:31.727410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.727417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.727419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.727421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.727423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9f3c110b-dffda47a-cbb2014-99ed3357_17"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 17
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.733087Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.733679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558268:3454], now have 1 active actors on pipe
2025-11-28T18:22:31.734987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.735004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.735077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 17 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.735129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 17 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.735139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 17 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.735144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 17 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.735162Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.735178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.735182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.735190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.735193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.735198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.735200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.735204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.735231Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 0
2025-11-28T18:22:31.735244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.735247Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 1
2025-11-28T18:22:31.735251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.735253Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 2
2025-11-28T18:22:31.735584Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 17 partNo 2 FormedBlobsCount 0 NewHead: Offset 17 PartNo 0 PackedSize 1048790 count 1 nextOffset 18 batches 3
2025-11-28T18:22:31.735594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.735597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.735600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.735668Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 17,1 HeadOffset 17 endOffset 17 curOffset 18 d0000000000_00000000000000000017_00000_0000000001_00002? size 1048780 WTime 1764354151735
2025-11-28T18:22:31.735843Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.735856Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 17 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.739563Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 17 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.739584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.739608Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.739616Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.739626Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 17, partNo: 0, Offset: 17 is stored on disk
2025-11-28T18:22:31.739630Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.739634Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 17, partNo: 1, Offset: 17 is stored on disk
2025-11-28T18:22:31.739641Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.739645Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 17, partNo: 2, Offset: 17 is stored on disk
2025-11-28T18:22:31.739680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.739683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.739685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.739687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.739689Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.739696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.739712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.739840Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.739909Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 17
    Offset: 17
    PartNo: 0
    WriteTimestampMS: 1764354151734
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:31.740399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558268:3454] destroyed
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.745994Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.746602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558274:3458], now have 1 active actors on pipe
2025-11-28T18:22:31.746950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.746960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.747002Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|62bc8d7c-a9b38634-fd15177c-32077f16_18 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.747014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.747018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.747024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.747027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.747034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.747045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.747049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.747054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.747074Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.747089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.747183Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.747662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558274:3458] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|62bc8d7c-a9b38634-fd15177c-32077f16_18"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|62bc8d7c-a9b38634-fd15177c-32077f16_18"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 18
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.755304Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.755909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558277:3459], now have 1 active actors on pipe
2025-11-28T18:22:31.757076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.757096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.757166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 18 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.757226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 18 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.757235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 18 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.757239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 18 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.757257Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.757275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.757278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.757287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.757291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.757296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.757297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.757302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.757331Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 0
2025-11-28T18:22:31.757345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.757348Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 1
2025-11-28T18:22:31.757351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.757354Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 2
2025-11-28T18:22:31.757860Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 18 partNo 2 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 1048790 count 1 nextOffset 19 batches 3
2025-11-28T18:22:31.757874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.757877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.757880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.757949Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 18,1 HeadOffset 18 endOffset 18 curOffset 19 d0000000000_00000000000000000018_00000_0000000001_00002? size 1048780 WTime 1764354151757
2025-11-28T18:22:31.758107Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.758118Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 18 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.761039Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 18 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.761068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.761082Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.761088Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.761095Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.761105Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 18, partNo: 0, Offset: 18 is stored on disk
2025-11-28T18:22:31.761108Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.761113Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 18, partNo: 1, Offset: 18 is stored on disk
2025-11-28T18:22:31.761118Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.761122Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 18, partNo: 2, Offset: 18 is stored on disk
2025-11-28T18:22:31.761151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.761154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.761156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.761157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.761160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.761169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.761182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.761254Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 18
    Offset: 18
    PartNo: 0
    WriteTimestampMS: 1764354151757
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:31.761587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558277:3459] destroyed
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.762123Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.762588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558282:3462], now have 1 active actors on pipe
2025-11-28T18:22:31.762664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.762672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.762708Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|8af92e39-b2962ce4-3c5f9c1a-49a9a30f_19 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.762716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.762719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.762724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.762728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.762731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.762741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.762744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.762747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.762760Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.762772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.762892Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.763247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558282:3462] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|8af92e39-b2962ce4-3c5f9c1a-49a9a30f_19"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|8af92e39-b2962ce4-3c5f9c1a-49a9a30f_19"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 19
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.772907Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.773484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558287:3465], now have 1 active actors on pipe
2025-11-28T18:22:31.774797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.774818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.774944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 19 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.775015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 19 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.775024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 19 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.775028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 19 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.775050Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.775073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.775077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.775090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.775093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.775097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.775100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.775105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.775134Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 0
2025-11-28T18:22:31.775149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.775152Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 1
2025-11-28T18:22:31.775155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.775158Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 2
2025-11-28T18:22:31.775394Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 19 partNo 2 FormedBlobsCount 0 NewHead: Offset 19 PartNo 0 PackedSize 1048790 count 1 nextOffset 20 batches 3
2025-11-28T18:22:31.775406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.775409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.775412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.775481Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 19,1 HeadOffset 19 endOffset 19 curOffset 20 d0000000000_00000000000000000019_00000_0000000001_00002? size 1048780 WTime 1764354151775
2025-11-28T18:22:31.775644Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.775658Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 19 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.779688Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 19 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.779710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.779734Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.779742Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.779752Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 19, partNo: 0, Offset: 19 is stored on disk
2025-11-28T18:22:31.779759Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.779763Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 19, partNo: 1, Offset: 19 is stored on disk
2025-11-28T18:22:31.779769Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.779773Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 19, partNo: 2, Offset: 19 is stored on disk
2025-11-28T18:22:31.779807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.779811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.779813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.779815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.779817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.779826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.779843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.779977Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.780063Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 19
    Offset: 19
    PartNo: 0
    WriteTimestampMS: 1764354151775
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.780982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558287:3465] destroyed
2025-11-28T18:22:31.781193Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.781773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558292:3468], now have 1 active actors on pipe
2025-11-28T18:22:31.781840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.781848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.781887Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|fe56e3be-8a87400f-73154a84-2c27e67b_20 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.781903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.781908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.781918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.781920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.781924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.781935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.781938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.781942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.781962Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.781981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.782056Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|fe56e3be-8a87400f-73154a84-2c27e67b_20"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2670
2025-11-28T18:22:31.782378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558292:3468] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|fe56e3be-8a87400f-73154a84-2c27e67b_20"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 20
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.789007Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.789783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558295:3469], now have 1 active actors on pipe
2025-11-28T18:22:31.791154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.791170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.791249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 20 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.791303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 20 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.791313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 20 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.791318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 20 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.791339Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.791359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.791368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.791378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.791380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.791384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.791387Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.791391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.791424Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 0
2025-11-28T18:22:31.791439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.791441Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 1
2025-11-28T18:22:31.791444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.791447Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 2
2025-11-28T18:22:31.791669Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 20 partNo 2 FormedBlobsCount 0 NewHead: Offset 20 PartNo 0 PackedSize 1048790 count 1 nextOffset 21 batches 3
2025-11-28T18:22:31.791689Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.791692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.791694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.791763Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 20,1 HeadOffset 20 endOffset 20 curOffset 21 d0000000000_00000000000000000020_00000_0000000001_00002? size 1048780 WTime 1764354151791
2025-11-28T18:22:31.791922Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.791934Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 20 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.795502Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 20 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.795534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.795562Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.795565Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.795574Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.795587Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 20, partNo: 0, Offset: 20 is stored on disk
2025-11-28T18:22:31.795590Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.795595Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 20, partNo: 1, Offset: 20 is stored on disk
2025-11-28T18:22:31.795603Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.795609Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 20, partNo: 2, Offset: 20 is stored on disk
2025-11-28T18:22:31.795648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.795655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.795657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.795660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.795662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.795674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.795691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.795809Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.796180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558295:3469] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 20
    Offset: 20
    PartNo: 0
    WriteTimestampMS: 1764354151791
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.796871Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.797379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558300:3472], now have 1 active actors on pipe
2025-11-28T18:22:31.797404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.797410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.797446Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|af757d5f-3bf20b48-c0ad588d-f89be429_21 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.797461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.797466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.797471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.797473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.797477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.797487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.797490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.797494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.797509Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.797526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.797588Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|af757d5f-3bf20b48-c0ad588d-f89be429_21"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|af757d5f-3bf20b48-c0ad588d-f89be429_21"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 21
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.801512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558300:3472] destroyed
2025-11-28T18:22:31.805040Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.806097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558303:3473], now have 1 active actors on pipe
2025-11-28T18:22:31.807311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.807334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.807465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 21 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.807571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 21 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.807582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 21 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.807587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 21 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.807607Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.807628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.807632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.807642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.807645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.807650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.807655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.807660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.807691Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 0
2025-11-28T18:22:31.807705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.807709Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 1
2025-11-28T18:22:31.807712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.807715Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 2
2025-11-28T18:22:31.808116Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 21 partNo 2 FormedBlobsCount 0 NewHead: Offset 21 PartNo 0 PackedSize 1048790 count 1 nextOffset 22 batches 3
2025-11-28T18:22:31.808127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.808130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.808133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.808201Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 21,1 HeadOffset 21 endOffset 21 curOffset 22 d0000000000_00000000000000000021_00000_0000000001_00002? size 1048780 WTime 1764354151807
2025-11-28T18:22:31.808358Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.808371Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 21 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.811466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 21 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.811494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.811514Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.811517Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.811522Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.811529Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 21, partNo: 0, Offset: 21 is stored on disk
2025-11-28T18:22:31.811533Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.811537Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 21, partNo: 1, Offset: 21 is stored on disk
2025-11-28T18:22:31.811543Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.811547Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 21, partNo: 2, Offset: 21 is stored on disk
2025-11-28T18:22:31.811576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.811578Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.811580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.811583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.811585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.811592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.811604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.811675Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 21
    Offset: 21
    PartNo: 0
    WriteTimestampMS: 1764354151807
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.812346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558303:3473] destroyed
2025-11-28T18:22:31.812614Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.813054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558308:3476], now have 1 active actors on pipe
2025-11-28T18:22:31.813198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.813206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.813287Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|d9c56f81-da3b47b5-11f04621-2585cba0_22 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.813299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.813302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.813306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.813309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.813312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.813320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.813323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.813325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.813337Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.813346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.813401Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|d9c56f81-da3b47b5-11f04621-2585cba0_22"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2670
2025-11-28T18:22:31.814313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558308:3476] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|d9c56f81-da3b47b5-11f04621-2585cba0_22"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 22
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.819962Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.820456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558311:3477], now have 1 active actors on pipe
2025-11-28T18:22:31.821602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.821622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.821719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 22 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.821846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 22 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.821856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 22 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.821862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 22 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.821885Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.821904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.821909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.821918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.821921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.821926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.821928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.821932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.821964Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 0
2025-11-28T18:22:31.821978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.821982Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 1
2025-11-28T18:22:31.821987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.821990Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 2
2025-11-28T18:22:31.822321Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 22 partNo 2 FormedBlobsCount 0 NewHead: Offset 22 PartNo 0 PackedSize 1048790 count 1 nextOffset 23 batches 3
2025-11-28T18:22:31.822331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.822335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.822338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.822406Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 22,1 HeadOffset 22 endOffset 22 curOffset 23 d0000000000_00000000000000000022_00000_0000000001_00002? size 1048780 WTime 1764354151822
2025-11-28T18:22:31.822677Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.822691Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 22 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.826092Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 22 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.826113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.826135Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.826141Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.826152Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 22, partNo: 0, Offset: 22 is stored on disk
2025-11-28T18:22:31.826156Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.826161Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 22, partNo: 1, Offset: 22 is stored on disk
2025-11-28T18:22:31.826166Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.826171Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 22, partNo: 2, Offset: 22 is stored on disk
2025-11-28T18:22:31.826202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.826205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.826207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.826210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.826212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.826220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.826236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.826355Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.826440Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.826829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558311:3477] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 22
    Offset: 22
    PartNo: 0
    WriteTimestampMS: 1764354151821
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.827483Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.827816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.827824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.827827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.827831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.827833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.828070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558316:3480], now have 1 active actors on pipe
2025-11-28T18:22:31.828085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.828090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.828121Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|746270fa-807e34b7-d574bfbe-a675cee0_23 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.828127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.828129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.828134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.828136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.828139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.828148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.828151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.828153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.828164Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.828174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.828307Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|746270fa-807e34b7-d574bfbe-a675cee0_23"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2670
2025-11-28T18:22:31.828588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558316:3480] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|746270fa-807e34b7-d574bfbe-a675cee0_23"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 23
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.834410Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.834996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558319:3481], now have 1 active actors on pipe
2025-11-28T18:22:31.836121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.836140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.836251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 23 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.836390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 23 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.836406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 23 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.836412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 23 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.836434Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.836456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.836463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.836474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.836480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.836485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.836488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.836493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.836524Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 0
2025-11-28T18:22:31.836545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.836552Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 1
2025-11-28T18:22:31.836559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.836566Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 2
2025-11-28T18:22:31.837070Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 23 partNo 2 FormedBlobsCount 0 NewHead: Offset 23 PartNo 0 PackedSize 1048790 count 1 nextOffset 24 batches 3
2025-11-28T18:22:31.837094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.837096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.837099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.837173Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 23,1 HeadOffset 23 endOffset 23 curOffset 24 d0000000000_00000000000000000023_00000_0000000001_00002? size 1048780 WTime 1764354151836
2025-11-28T18:22:31.837521Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.837543Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 23 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.840889Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 23 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.840912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.840988Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.842386Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:496: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-11-28T18:22:31.842421Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.842434Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.842446Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 23, partNo: 0, Offset: 23 is stored on disk
2025-11-28T18:22:31.842454Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.842474Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 23, partNo: 1, Offset: 23 is stored on disk
2025-11-28T18:22:31.842479Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.842484Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 23, partNo: 2, Offset: 23 is stored on disk
2025-11-28T18:22:31.842521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.842530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.842534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.842536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.842538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.842551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000016_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.842560Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000017_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.842564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000018_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.842566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000019_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.842569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.842575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.842577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.842579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000023_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.842582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 8 keys were taken away. Let's read 8390240 bytes
2025-11-28T18:22:31.842605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.842626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 8 blobs
2025-11-28T18:22:31.842633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000016_00000_0000000001_00002?
2025-11-28T18:22:31.842644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000017_00000_0000000001_00002?
2025-11-28T18:22:31.842655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000018_00000_0000000001_00002?
2025-11-28T18:22:31.842669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000019_00000_0000000001_00002?
2025-11-28T18:22:31.842679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00000_0000000001_00002?
2025-11-28T18:22:31.842728Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.842700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00002?
2025-11-28T18:22:31.842714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000001_00002?
2025-11-28T18:22:31.842724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000023_00000_0000000001_00002?
2025-11-28T18:22:31.842736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 8 blobs for compaction
2025-11-28T18:22:31.842747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 16 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.842751Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 17 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.842756Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 18 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.842760Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 19 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.842764Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 20 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.842768Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 21 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.842772Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 22 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.842775Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 23 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.842788Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 8 blobs are from cache.
2025-11-28T18:22:31.842820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-28T18:22:31.842829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/8] d0000000000_00000000000000000016_00000_0000000001_00002?
2025-11-28T18:22:31.842831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 23
    Offset: 23
    PartNo: 0
    WriteTimestampMS: 1764354151836
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:31.842834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000016_00000_0000000001_00002?, parameters.CurOffset=16
2025-11-28T18:22:31.842987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 16.0/3
2025-11-28T18:22:31.842995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 0
2025-11-28T18:22:31.843000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 16:0 appended
2025-11-28T18:22:31.843102Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.843106Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.843110Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.843113Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.843116Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.843120Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.843123Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.843126Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.843156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 16.1/3
2025-11-28T18:22:31.843159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 1
2025-11-28T18:22:31.843161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 16:1 appended
2025-11-28T18:22:31.843186Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 16.2/3
2025-11-28T18:22:31.843189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 2
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.843400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 16 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 1048790 count 1 nextOffset 17 batches 3
2025-11-28T18:22:31.843418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 16:2 appended
2025-11-28T18:22:31.843422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/8] d0000000000_00000000000000000017_00000_0000000001_00002?
2025-11-28T18:22:31.843424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.843427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000017_00000_0000000001_00002?, parameters.CurOffset=17
2025-11-28T18:22:31.843582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 17.0/3
2025-11-28T18:22:31.843595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 0
2025-11-28T18:22:31.843598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 17:0 appended
2025-11-28T18:22:31.843753Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 17.1/3
2025-11-28T18:22:31.843763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 1
2025-11-28T18:22:31.843765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 17:1 appended
2025-11-28T18:22:31.843780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 17.2/3
2025-11-28T18:22:31.843786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 2
2025-11-28T18:22:31.843894Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.843986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 17 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 2097545 count 2 nextOffset 18 batches 5
2025-11-28T18:22:31.843999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 17:2 appended
2025-11-28T18:22:31.844003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/8] d0000000000_00000000000000000018_00000_0000000001_00002?
2025-11-28T18:22:31.844005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.844007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000018_00000_0000000001_00002?, parameters.CurOffset=18
2025-11-28T18:22:31.844154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 18.0/3
2025-11-28T18:22:31.844165Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 0
2025-11-28T18:22:31.844167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 18:0 appended
2025-11-28T18:22:31.844318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 18.1/3
2025-11-28T18:22:31.844331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 1
2025-11-28T18:22:31.844333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 18:1 appended
2025-11-28T18:22:31.844348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 18.2/3
2025-11-28T18:22:31.844350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 2
2025-11-28T18:22:31.844543Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 18 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 3146300 count 3 nextOffset 19 batches 7
2025-11-28T18:22:31.844557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 18:2 appended
2025-11-28T18:22:31.844561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/8] d0000000000_00000000000000000019_00000_0000000001_00002?
2025-11-28T18:22:31.844562Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.844565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000019_00000_0000000001_00002?, parameters.CurOffset=19
2025-11-28T18:22:31.844902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 19.0/3
2025-11-28T18:22:31.844915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 0
2025-11-28T18:22:31.844919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 19:0 appended
2025-11-28T18:22:31.845061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 19.1/3
2025-11-28T18:22:31.845071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 1
2025-11-28T18:22:31.845074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 19:1 appended
2025-11-28T18:22:31.845086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 19.2/3
2025-11-28T18:22:31.845088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 2
2025-11-28T18:22:31.845659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 19 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 4195055 count 4 nextOffset 20 batches 9
2025-11-28T18:22:31.845675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 19:2 appended
2025-11-28T18:22:31.845680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/8] d0000000000_00000000000000000020_00000_0000000001_00002?
2025-11-28T18:22:31.845682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.845685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000020_00000_0000000001_00002?, parameters.CurOffset=20
2025-11-28T18:22:31.846172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 20.0/3
2025-11-28T18:22:31.846188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 0
2025-11-28T18:22:31.846192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 20:0 appended
2025-11-28T18:22:31.847356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 20.1/3
2025-11-28T18:22:31.847371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 1
2025-11-28T18:22:31.847374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 20:1 appended
2025-11-28T18:22:31.847392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 20.2/3
2025-11-28T18:22:31.847394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 2
2025-11-28T18:22:31.847569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 20 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 5243810 count 5 nextOffset 21 batches 11
2025-11-28T18:22:31.847586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 20:2 appended
2025-11-28T18:22:31.847590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/8] d0000000000_00000000000000000021_00000_0000000001_00002?
2025-11-28T18:22:31.847592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.847597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000021_00000_0000000001_00002?, parameters.CurOffset=21
2025-11-28T18:22:31.847727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 21.0/3
2025-11-28T18:22:31.847741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 0
2025-11-28T18:22:31.847744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 21:0 appended
2025-11-28T18:22:31.847891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 21.1/3
2025-11-28T18:22:31.847900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 1
2025-11-28T18:22:31.847902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 21:1 appended
2025-11-28T18:22:31.847912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 21.2/3
2025-11-28T18:22:31.847915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 2
2025-11-28T18:22:31.848509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 21 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 6292565 count 6 nextOffset 22 batches 13
2025-11-28T18:22:31.848522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 21:2 appended
2025-11-28T18:22:31.848526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/8] d0000000000_00000000000000000022_00000_0000000001_00002?
2025-11-28T18:22:31.848529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.848532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000022_00000_0000000001_00002?, parameters.CurOffset=22
2025-11-28T18:22:31.848678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 22.0/3
2025-11-28T18:22:31.848690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 0
2025-11-28T18:22:31.848693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 22:0 appended
2025-11-28T18:22:31.849192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 22.1/3
2025-11-28T18:22:31.849207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 1
2025-11-28T18:22:31.849210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 22:1 appended
2025-11-28T18:22:31.849223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 22.2/3
2025-11-28T18:22:31.849226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 2
2025-11-28T18:22:31.849787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 22 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 7341320 count 7 nextOffset 23 batches 15
2025-11-28T18:22:31.849803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 22:2 appended
2025-11-28T18:22:31.849807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/8] d0000000000_00000000000000000023_00000_0000000001_00002?
2025-11-28T18:22:31.849809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.849812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000023_00000_0000000001_00002?, parameters.CurOffset=23
2025-11-28T18:22:31.849961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 23.0/3
2025-11-28T18:22:31.849973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 0
2025-11-28T18:22:31.857461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 23 partNo 0 result is x0000000000_00000000000000000015_00001_0000000008_00015 size 7878063
2025-11-28T18:22:31.857487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 23:0 appended
2025-11-28T18:22:31.857958Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 23.1/3
2025-11-28T18:22:31.857975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 1
2025-11-28T18:22:31.857979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 23:1 appended
2025-11-28T18:22:31.857999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 23.2/3
2025-11-28T18:22:31.858002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 2
2025-11-28T18:22:31.858018Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1110: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000015_00001_0000000008_00015 new key d0000000000_00000000000000000015_00001_0000000008_00015 size 7878063 WTime 1764354151854
2025-11-28T18:22:31.858244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 23 partNo 2 FormedBlobsCount 1 NewHead: Offset 23 PartNo 0 PackedSize 1048790 count 1 nextOffset 24 batches 3
2025-11-28T18:22:31.858254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 23:2 appended
2025-11-28T18:22:31.858313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 23,1 HeadOffset 15 endOffset 16 curOffset 24 d0000000000_00000000000000000023_00000_0000000001_00002| size 1048780 WTime 1764354151854
2025-11-28T18:22:31.858599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558319:3481] destroyed
2025-11-28T18:22:31.858613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558325:3485], now have 1 active actors on pipe
2025-11-28T18:22:31.858634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.858642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.858658Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.858670Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 15 partNo 1 count 8 size 7878063
2025-11-28T18:22:31.858678Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 23 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.858713Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|dd302386-c1e41389-d6a033ca-24464f30_24 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.858723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.858726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.858732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.858735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.858740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.858750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.858753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.858756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.858771Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.858821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.859327Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.859979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558325:3485] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|dd302386-c1e41389-d6a033ca-24464f30_24"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|dd302386-c1e41389-d6a033ca-24464f30_24"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 24
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.869895Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.870409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558330:3488], now have 1 active actors on pipe
2025-11-28T18:22:31.870978Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 15 count 8 size 7878063 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.870990Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 23 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.871008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.871085Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 15 partno 1 count 8 parts 15 suffix '0' size 7878063
2025-11-28T18:22:31.871091Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 7 parts 16 suffix '0' size 8365339
2025-11-28T18:22:31.871095Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 2 suffix '124' size 1048780
2025-11-28T18:22:31.871253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.871257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.871259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.871262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.871264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.871302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.871313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 0 size 8365339 cause it's been evicted from L2. Actual L1 size: 12
2025-11-28T18:22:31.871318Z node 2 :PERSQUEUE NOTICE: read.h:372: [72075186224037892][PQCacheProxy]Have to remove new data from cache. Topic rt3.dc1--topic1, tablet id72075186224037892, cookie 0
2025-11-28T18:22:31.871630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.871638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.871705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 24 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.871751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 24 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.871762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 24 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.871766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 24 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.871781Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.871789Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000016_00000_0000000001_00002?(+) to d0000000000_00000000000000000016_00000_0000000001_00002?(+)
2025-11-28T18:22:31.871791Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000017_00000_0000000001_00002?(+) to d0000000000_00000000000000000017_00000_0000000001_00002?(+)
2025-11-28T18:22:31.871795Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000018_00000_0000000001_00002?(+) to d0000000000_00000000000000000018_00000_0000000001_00002?(+)
2025-11-28T18:22:31.871797Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000019_00000_0000000001_00002?(+) to d0000000000_00000000000000000019_00000_0000000001_00002?(+)
2025-11-28T18:22:31.871800Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000020_00000_0000000001_00002?(+) to d0000000000_00000000000000000020_00000_0000000001_00002?(+)
2025-11-28T18:22:31.871803Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000021_00000_0000000001_00002?(+) to d0000000000_00000000000000000021_00000_0000000001_00002?(+)
2025-11-28T18:22:31.871805Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000022_00000_0000000001_00002?(+) to d0000000000_00000000000000000022_00000_0000000001_00002?(+)
2025-11-28T18:22:31.871808Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000023_00000_0000000001_00002?(+) to d0000000000_00000000000000000023_00000_0000000001_00002?(+)
2025-11-28T18:22:31.871810Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000015_00001_0000000001_00001|(+) to d0000000000_00000000000000000015_00001_0000000001_00001|(+)
2025-11-28T18:22:31.871819Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.871834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-28T18:22:31.872821Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 16 count 1 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.872832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 17 count 1 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.872837Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 18 count 1 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.872842Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 19 count 1 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.872849Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 20 count 1 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.872853Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 21 count 1 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.872856Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 22 count 1 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.872860Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 23 count 1 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.872865Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 15 count 1 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.872885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.872901Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.872903Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.872910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.872915Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.872923Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.872930Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.872936Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.872941Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.872946Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.872946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.872951Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.872956Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 15 partno 1 count 1 parts 1 suffix '124' size 536753
2025-11-28T18:22:31.872957Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.872960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.872964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.872966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.872970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.872992Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 0
2025-11-28T18:22:31.873010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.873013Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 1
2025-11-28T18:22:31.873017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.873020Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 2
2025-11-28T18:22:31.873249Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 24 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 1048790 count 1 nextOffset 25 batches 3
2025-11-28T18:22:31.873272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.873275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.873278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.873341Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 24,1 HeadOffset 24 endOffset 24 curOffset 25 d0000000000_00000000000000000024_00000_0000000001_00002? size 1048780 WTime 1764354151873
2025-11-28T18:22:31.873497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.873523Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.873539Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 24 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.877730Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 24 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.877762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.877784Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.877788Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.877796Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.877806Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 24, partNo: 0, Offset: 24 is stored on disk
2025-11-28T18:22:31.877815Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.877820Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 24, partNo: 1, Offset: 24 is stored on disk
2025-11-28T18:22:31.877826Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.877836Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 24, partNo: 2, Offset: 24 is stored on disk
2025-11-28T18:22:31.877874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.877887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.877894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.877897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.877899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.877909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.877926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.878003Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.878297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558330:3488] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 24
    Offset: 24
    PartNo: 0
    WriteTimestampMS: 1764354151872
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.879102Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.879552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558338:3494], now have 1 active actors on pipe
2025-11-28T18:22:31.879757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.879766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.879804Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|14d0dcd0-e810fb48-c81f743a-e595a261_25 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.879813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.879816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.879821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.879823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.879827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.879837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.879840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.879843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.879859Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.879871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.879915Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.880103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558338:3494] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|14d0dcd0-e810fb48-c81f743a-e595a261_25"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|14d0dcd0-e810fb48-c81f743a-e595a261_25"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 25
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.886937Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.887448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558341:3495], now have 1 active actors on pipe
2025-11-28T18:22:31.888469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.888488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.888560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 25 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.888622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 25 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.888643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 25 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.888652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 25 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.888676Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.888701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.888708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.888719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.888735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.888740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.888744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.888749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.888781Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 0
2025-11-28T18:22:31.888802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.888809Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 1
2025-11-28T18:22:31.888813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.888819Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 2
2025-11-28T18:22:31.889111Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 25 partNo 2 FormedBlobsCount 0 NewHead: Offset 25 PartNo 0 PackedSize 1048790 count 1 nextOffset 26 batches 3
2025-11-28T18:22:31.889127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.889134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.889138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.889211Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 25,1 HeadOffset 25 endOffset 25 curOffset 26 d0000000000_00000000000000000025_00000_0000000001_00002? size 1048780 WTime 1764354151889
2025-11-28T18:22:31.889372Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.889392Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 25 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.892145Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 25 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.892173Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.892189Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.892198Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.892218Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.892230Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 25, partNo: 0, Offset: 25 is stored on disk
2025-11-28T18:22:31.892238Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.892243Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 25, partNo: 1, Offset: 25 is stored on disk
2025-11-28T18:22:31.892248Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.892252Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 25, partNo: 2, Offset: 25 is stored on disk
2025-11-28T18:22:31.892288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.892296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.892298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.892301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.892303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.892312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.892329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.892436Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.892741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558341:3495] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 25
    Offset: 25
    PartNo: 0
    WriteTimestampMS: 1764354151888
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.893476Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.893834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558346:3498], now have 1 active actors on pipe
2025-11-28T18:22:31.893933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.893943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.893975Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|48a55b02-79bea8c0-64510b30-992819d6_26 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.893987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.893991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.893996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.894002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.894005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.894014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.894017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.894022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.894039Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.894054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.894115Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.894301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558346:3498] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|48a55b02-79bea8c0-64510b30-992819d6_26"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|48a55b02-79bea8c0-64510b30-992819d6_26"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 26
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.901037Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.901500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558349:3499], now have 1 active actors on pipe
2025-11-28T18:22:31.902751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.902771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.902847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 26 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.902910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 26 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.902925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 26 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.902934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 26 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.902958Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.902983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.902987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.902996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.903000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.903010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.903012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.903020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.903055Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 0
2025-11-28T18:22:31.903072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.903080Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 1
2025-11-28T18:22:31.903084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.903087Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 2
2025-11-28T18:22:31.903385Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 26 partNo 2 FormedBlobsCount 0 NewHead: Offset 26 PartNo 0 PackedSize 1048790 count 1 nextOffset 27 batches 3
2025-11-28T18:22:31.903403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.903407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.903414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.903489Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 26,1 HeadOffset 26 endOffset 26 curOffset 27 d0000000000_00000000000000000026_00000_0000000001_00002? size 1048780 WTime 1764354151903
2025-11-28T18:22:31.903658Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.903679Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 26 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.906523Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 26 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.906547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.906570Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.906576Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.906577Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.906587Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 26, partNo: 0, Offset: 26 is stored on disk
2025-11-28T18:22:31.906591Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.906594Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 26, partNo: 1, Offset: 26 is stored on disk
2025-11-28T18:22:31.906600Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.906606Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 26, partNo: 2, Offset: 26 is stored on disk
2025-11-28T18:22:31.906641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.906647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.906649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.906652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.906654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.906665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.906684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.906753Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.907026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558349:3499] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 26
    Offset: 26
    PartNo: 0
    WriteTimestampMS: 1764354151903
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.907796Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.908263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558355:3503], now have 1 active actors on pipe
2025-11-28T18:22:31.908348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.908361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.908396Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|87d3e235-a18bf300-361d7672-ae7ba14_27 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.908410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.908416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.908421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.908424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.908427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.908435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.908439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.908442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.908456Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.908471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.908520Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|87d3e235-a18bf300-361d7672-ae7ba14_27"
    Status: Active
    SeqNo: 0
  }
}
2025-11-28T18:22:31.908770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558355:3503] destroyed
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|87d3e235-a18bf300-361d7672-ae7ba14_27"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 27
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.915413Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.915988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558358:3504], now have 1 active actors on pipe
2025-11-28T18:22:31.917205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.917225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.917313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 27 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.917367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 27 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.917386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 27 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.917392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 27 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.917414Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.917438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.917446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.917458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.917465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.917470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.917474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.917478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.917514Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 0
2025-11-28T18:22:31.917532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.917539Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 1
2025-11-28T18:22:31.917543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.917546Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 2
2025-11-28T18:22:31.917763Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 27 partNo 2 FormedBlobsCount 0 NewHead: Offset 27 PartNo 0 PackedSize 1048790 count 1 nextOffset 28 batches 3
2025-11-28T18:22:31.917780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.917783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.917788Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.917858Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 27,1 HeadOffset 27 endOffset 27 curOffset 28 d0000000000_00000000000000000027_00000_0000000001_00002? size 1048780 WTime 1764354151917
2025-11-28T18:22:31.918173Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.918192Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 27 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.920975Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 27 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.920998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.921021Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.921022Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.921032Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.921042Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 27, partNo: 0, Offset: 27 is stored on disk
2025-11-28T18:22:31.921049Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.921054Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 27, partNo: 1, Offset: 27 is stored on disk
2025-11-28T18:22:31.921061Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.921065Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 27, partNo: 2, Offset: 27 is stored on disk
2025-11-28T18:22:31.921098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.921105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.921108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.921110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.921112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.921121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.921137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.921216Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.921494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558358:3504] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 27
    Offset: 27
    PartNo: 0
    WriteTimestampMS: 1764354151917
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.922071Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.922425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558364:3507], now have 1 active actors on pipe
2025-11-28T18:22:31.922503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.922513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.922546Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|8b4a663e-ce7b1820-da1af253-e320cb9c_28 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.922558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.922562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.922566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.922573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.922576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.922588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.922590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.922593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.922604Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.922618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.922666Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.922830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558364:3507] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|8b4a663e-ce7b1820-da1af253-e320cb9c_28"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|8b4a663e-ce7b1820-da1af253-e320cb9c_28"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 28
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.927263Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.927794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558367:3508], now have 1 active actors on pipe
2025-11-28T18:22:31.928116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.928127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.928130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.928133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.928135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.928951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.928973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.929093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 28 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.929209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 28 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.929225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 28 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.929234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 28 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.929260Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.929280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.929284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.929294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.929301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.929304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.929306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.929309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.929338Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 0
2025-11-28T18:22:31.929357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.929364Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 1
2025-11-28T18:22:31.929368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.929374Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 2
2025-11-28T18:22:31.929594Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 28 partNo 2 FormedBlobsCount 0 NewHead: Offset 28 PartNo 0 PackedSize 1048790 count 1 nextOffset 29 batches 3
2025-11-28T18:22:31.929611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.929618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.929624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.929697Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 28,1 HeadOffset 28 endOffset 28 curOffset 29 d0000000000_00000000000000000028_00000_0000000001_00002? size 1048780 WTime 1764354151929
2025-11-28T18:22:31.929860Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.929882Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 28 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.932779Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 28 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.932805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.932827Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.932831Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.932838Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.932848Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 28, partNo: 0, Offset: 28 is stored on disk
2025-11-28T18:22:31.932855Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.932859Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 28, partNo: 1, Offset: 28 is stored on disk
2025-11-28T18:22:31.932865Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.932868Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 28, partNo: 2, Offset: 28 is stored on disk
2025-11-28T18:22:31.932903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.932911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.932913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.932915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.932917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.932929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.932950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.933045Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.933376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558367:3508] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 28
    Offset: 28
    PartNo: 0
    WriteTimestampMS: 1764354151929
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.934127Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.934513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558373:3512], now have 1 active actors on pipe
2025-11-28T18:22:31.934608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.934620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.934663Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|1dba916e-81733351-b0d1684-ecd99ae4_29 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.934676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.934684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.934688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.934690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.934708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.934720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.934727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.934730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.934744Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.934759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.934860Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.935108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558373:3512] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|1dba916e-81733351-b0d1684-ecd99ae4_29"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|1dba916e-81733351-b0d1684-ecd99ae4_29"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 29
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.940509Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.941003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558377:3514], now have 1 active actors on pipe
2025-11-28T18:22:31.942308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.942328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.942427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 29 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.942502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 29 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.942521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 29 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.942533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 29 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.942556Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.942577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.942584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.942595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.942598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.942602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.942609Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.942614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.942641Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 0
2025-11-28T18:22:31.942657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.942664Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 1
2025-11-28T18:22:31.942668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.942671Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 2
2025-11-28T18:22:31.942923Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 29 partNo 2 FormedBlobsCount 0 NewHead: Offset 29 PartNo 0 PackedSize 1048790 count 1 nextOffset 30 batches 3
2025-11-28T18:22:31.942939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.942942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.942945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.943013Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 29,1 HeadOffset 29 endOffset 29 curOffset 30 d0000000000_00000000000000000029_00000_0000000001_00002? size 1048780 WTime 1764354151942
2025-11-28T18:22:31.943317Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.943336Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 29 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.946378Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 29 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.946408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.946431Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.946430Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.946439Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.946449Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 29, partNo: 0, Offset: 29 is stored on disk
2025-11-28T18:22:31.946458Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.946484Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 29, partNo: 1, Offset: 29 is stored on disk
2025-11-28T18:22:31.946494Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.946499Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 29, partNo: 2, Offset: 29 is stored on disk
2025-11-28T18:22:31.946539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.946546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.946549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.946552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.946555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.946564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.946578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.946672Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.946996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558377:3514] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 29
    Offset: 29
    PartNo: 0
    WriteTimestampMS: 1764354151942
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.947701Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.948063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558382:3517], now have 1 active actors on pipe
2025-11-28T18:22:31.948154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.948163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.948198Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|1ead2b63-680d1959-4df104f1-cf132789_30 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.948206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.948209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.948213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.948215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.948219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.948227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.948230Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.948235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.948246Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.948256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.948314Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|1ead2b63-680d1959-4df104f1-cf132789_30"
    Status: Active
    SeqNo: 0
  }
}
2025-11-28T18:22:31.948544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558382:3517] destroyed
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|1ead2b63-680d1959-4df104f1-cf132789_30"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 30
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.954180Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.954581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558388:3521], now have 1 active actors on pipe
2025-11-28T18:22:31.955763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.955784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.955858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 30 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.955915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 30 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.955930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 30 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.955939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 30 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.955960Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.955985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.955992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.956002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.956005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.956008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.956010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.956015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.956049Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 0
2025-11-28T18:22:31.956069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.956072Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 1
2025-11-28T18:22:31.956076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.956079Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 2
2025-11-28T18:22:31.956661Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 30 partNo 2 FormedBlobsCount 0 NewHead: Offset 30 PartNo 0 PackedSize 1048790 count 1 nextOffset 31 batches 3
2025-11-28T18:22:31.956681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.956685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.956688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.956777Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 30,1 HeadOffset 30 endOffset 30 curOffset 31 d0000000000_00000000000000000030_00000_0000000001_00002? size 1048780 WTime 1764354151956
2025-11-28T18:22:31.956970Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.956990Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 30 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.959939Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 30 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.959968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.959979Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.959994Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.960001Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.960010Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 30, partNo: 0, Offset: 30 is stored on disk
2025-11-28T18:22:31.960013Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.960017Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 30, partNo: 1, Offset: 30 is stored on disk
2025-11-28T18:22:31.960023Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.960031Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 30, partNo: 2, Offset: 30 is stored on disk
2025-11-28T18:22:31.960062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.960069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.960071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.960074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.960076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.960086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.960107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.960265Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 30
    Offset: 30
    PartNo: 0
    WriteTimestampMS: 1764354151955
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:31.960605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558388:3521] destroyed
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.961184Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.961558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558393:3524], now have 1 active actors on pipe
2025-11-28T18:22:31.961660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.961674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.961708Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|e63139c3-11d35262-2a9c71f9-6db7ccf_31 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.961724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.961728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.961733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.961735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.961739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.961748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.961755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.961758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.961772Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.961786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.961832Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.962004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558393:3524] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|e63139c3-11d35262-2a9c71f9-6db7ccf_31"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|e63139c3-11d35262-2a9c71f9-6db7ccf_31"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 31
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:31.967566Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.968005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558396:3525], now have 1 active actors on pipe
2025-11-28T18:22:31.969732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.969753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.969837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 31 partNo : 0 messageNo: 0 size: 511959
2025-11-28T18:22:31.969905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 31 partNo : 1 messageNo: 0 size: 511959
2025-11-28T18:22:31.969929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 31 partNo : 2 messageNo: 0 size: 24658
2025-11-28T18:22:31.969934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 31 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-28T18:22:31.969959Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.969981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.969989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.969998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.970002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.970006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-28T18:22:31.970008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.970014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.970049Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 0
2025-11-28T18:22:31.970063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.970067Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 1
2025-11-28T18:22:31.970070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.970073Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 2
2025-11-28T18:22:31.970296Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 31 partNo 2 FormedBlobsCount 0 NewHead: Offset 31 PartNo 0 PackedSize 1048790 count 1 nextOffset 32 batches 3
2025-11-28T18:22:31.970317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-28T18:22:31.970320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-28T18:22:31.970325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.970402Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 31,1 HeadOffset 31 endOffset 31 curOffset 32 d0000000000_00000000000000000031_00000_0000000001_00002? size 1048780 WTime 1764354151970
2025-11-28T18:22:31.970621Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.970643Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 31 partNo 0 count 1 size 1048780
2025-11-28T18:22:31.973537Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 31 count 1 size 1048780 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.973834Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.973588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.973597Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:31.973620Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.973627Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.973638Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 31, partNo: 0, Offset: 31 is stored on disk
2025-11-28T18:22:31.973642Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.973646Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 31, partNo: 1, Offset: 31 is stored on disk
2025-11-28T18:22:31.973652Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.973656Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 31, partNo: 2, Offset: 31 is stored on disk
2025-11-28T18:22:31.973692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.973695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.973697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.973700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.973703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.973715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000024_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.973719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000025_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.973723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000026_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.973726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000027_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.973729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000028_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.973732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000029_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.973734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000030_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.973736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000031_00000_0000000001_00002? (1048780)
2025-11-28T18:22:31.973739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 8 keys were taken away. Let's read 8390240 bytes
2025-11-28T18:22:31.973757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.973771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 8 blobs
2025-11-28T18:22:31.973774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000024_00000_0000000001_00002?
2025-11-28T18:22:31.973785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000025_00000_0000000001_00002?
2025-11-28T18:22:31.973797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000026_00000_0000000001_00002?
2025-11-28T18:22:31.973811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000027_00000_0000000001_00002?
2025-11-28T18:22:31.973821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000028_00000_0000000001_00002?
2025-11-28T18:22:31.973840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000029_00000_0000000001_00002?
2025-11-28T18:22:31.973849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000030_00000_0000000001_00002?
2025-11-28T18:22:31.973859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000031_00000_0000000001_00002?
2025-11-28T18:22:31.973868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 8 blobs for compaction
CallPersQueueGRPC response:
2025-11-28T18:22:31.973878Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 24 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.973882Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 25 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.973886Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 26 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.973892Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 27 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.973895Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 28 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.973899Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 29 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.973902Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 30 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.973905Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 31 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 31
    Offset: 31
    PartNo: 0
    WriteTimestampMS: 1764354151969
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:31.973920Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 8 blobs are from cache.
2025-11-28T18:22:31.973949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-28T18:22:31.973957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/8] d0000000000_00000000000000000024_00000_0000000001_00002?
2025-11-28T18:22:31.973960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.973963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000024_00000_0000000001_00002?, parameters.CurOffset=24
2025-11-28T18:22:31.974092Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.974098Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.974103Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.974107Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.974111Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.974112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 24.0/3
2025-11-28T18:22:31.974114Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.974118Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 2 suffix '63'
>>>>> 1
2025-11-28T18:22:31.974120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 0
2025-11-28T18:22:31.974122Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 2 suffix '63'
2025-11-28T18:22:31.974127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 24:0 appended
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdRead {
    ClientId: "user"
    Offset: 0
    Count: 16
    ReadTimestampMs: 0
  }
}
2025-11-28T18:22:31.974276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 24.1/3
2025-11-28T18:22:31.974294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 1
2025-11-28T18:22:31.974297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 24:1 appended
2025-11-28T18:22:31.974321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 24.2/3
2025-11-28T18:22:31.974324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 2
2025-11-28T18:22:31.974832Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.974561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 24 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 1048790 count 1 nextOffset 25 batches 3
2025-11-28T18:22:31.974576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 24:2 appended
2025-11-28T18:22:31.974581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/8] d0000000000_00000000000000000025_00000_0000000001_00002?
2025-11-28T18:22:31.974583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.974586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000025_00000_0000000001_00002?, parameters.CurOffset=25
2025-11-28T18:22:31.974718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 25.0/3
2025-11-28T18:22:31.974723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 0
2025-11-28T18:22:31.974726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 25:0 appended
2025-11-28T18:22:31.974861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 25.1/3
2025-11-28T18:22:31.974868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 1
2025-11-28T18:22:31.974872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 25:1 appended
2025-11-28T18:22:31.974885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 25.2/3
2025-11-28T18:22:31.974888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 2
2025-11-28T18:22:31.975073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 25 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 2097545 count 2 nextOffset 26 batches 5
2025-11-28T18:22:31.975078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 25:2 appended
2025-11-28T18:22:31.975082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/8] d0000000000_00000000000000000026_00000_0000000001_00002?
2025-11-28T18:22:31.975084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.975087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000026_00000_0000000001_00002?, parameters.CurOffset=26
2025-11-28T18:22:31.975223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 26.0/3
2025-11-28T18:22:31.975228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 0
2025-11-28T18:22:31.975230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 26:0 appended
2025-11-28T18:22:31.975751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 26.1/3
2025-11-28T18:22:31.975758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 1
2025-11-28T18:22:31.975761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 26:1 appended
2025-11-28T18:22:31.975782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 26.2/3
2025-11-28T18:22:31.975785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 2
2025-11-28T18:22:31.975971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 26 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 3146300 count 3 nextOffset 27 batches 7
2025-11-28T18:22:31.975977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 26:2 appended
2025-11-28T18:22:31.975982Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/8] d0000000000_00000000000000000027_00000_0000000001_00002?
2025-11-28T18:22:31.975984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.975988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000027_00000_0000000001_00002?, parameters.CurOffset=27
2025-11-28T18:22:31.976150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 27.0/3
2025-11-28T18:22:31.976157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 0
2025-11-28T18:22:31.976160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 27:0 appended
2025-11-28T18:22:31.976300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 27.1/3
2025-11-28T18:22:31.976305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 1
2025-11-28T18:22:31.976308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 27:1 appended
2025-11-28T18:22:31.976320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 27.2/3
2025-11-28T18:22:31.976323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 2
2025-11-28T18:22:31.976924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 27 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 4195055 count 4 nextOffset 28 batches 9
2025-11-28T18:22:31.976943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 27:2 appended
2025-11-28T18:22:31.976947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/8] d0000000000_00000000000000000028_00000_0000000001_00002?
2025-11-28T18:22:31.976951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.976954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000028_00000_0000000001_00002?, parameters.CurOffset=28
2025-11-28T18:22:31.977109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 28.0/3
2025-11-28T18:22:31.977124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 0
2025-11-28T18:22:31.977127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 28:0 appended
2025-11-28T18:22:31.977665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 28.1/3
2025-11-28T18:22:31.977679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 1
2025-11-28T18:22:31.977681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 28:1 appended
2025-11-28T18:22:31.977698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 28.2/3
2025-11-28T18:22:31.977701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 2
2025-11-28T18:22:31.977880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 28 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 5243810 count 5 nextOffset 29 batches 11
2025-11-28T18:22:31.977896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 28:2 appended
2025-11-28T18:22:31.977900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/8] d0000000000_00000000000000000029_00000_0000000001_00002?
2025-11-28T18:22:31.977902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.977905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000029_00000_0000000001_00002?, parameters.CurOffset=29
2025-11-28T18:22:31.978057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 29.0/3
2025-11-28T18:22:31.978075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 0
2025-11-28T18:22:31.978078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 29:0 appended
2025-11-28T18:22:31.978214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 29.1/3
2025-11-28T18:22:31.978228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 1
2025-11-28T18:22:31.978231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 29:1 appended
2025-11-28T18:22:31.978245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 29.2/3
2025-11-28T18:22:31.978247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 2
2025-11-28T18:22:31.978430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 29 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 6292565 count 6 nextOffset 30 batches 13
2025-11-28T18:22:31.978447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 29:2 appended
2025-11-28T18:22:31.978451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/8] d0000000000_00000000000000000030_00000_0000000001_00002?
2025-11-28T18:22:31.978454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.978457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000030_00000_0000000001_00002?, parameters.CurOffset=30
2025-11-28T18:22:31.978618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 30.0/3
2025-11-28T18:22:31.978631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 0
2025-11-28T18:22:31.978636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 30:0 appended
2025-11-28T18:22:31.978784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 30.1/3
2025-11-28T18:22:31.978797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 1
2025-11-28T18:22:31.978799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 30:1 appended
2025-11-28T18:22:31.978812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 30.2/3
2025-11-28T18:22:31.978820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 2
2025-11-28T18:22:31.982804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 30 partNo 2 result is x0000000000_00000000000000000023_00000_0000000007_00016 size 8365364
2025-11-28T18:22:31.982833Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1110: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000023_00000_0000000007_00016 new key d0000000000_00000000000000000023_00000_0000000007_00016 size 8365364 WTime 1764354151982
2025-11-28T18:22:31.982872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 30 partNo 2 FormedBlobsCount 1 NewHead: Offset 30 PartNo 2 PackedSize 24736 count 1 nextOffset 31 batches 1
2025-11-28T18:22:31.982886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 30:2 appended
2025-11-28T18:22:31.982892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/8] d0000000000_00000000000000000031_00000_0000000001_00002?
2025-11-28T18:22:31.982900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.982904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000031_00000_0000000001_00002?, parameters.CurOffset=31
2025-11-28T18:22:31.983036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 31.0/3
2025-11-28T18:22:31.983055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 0
2025-11-28T18:22:31.983059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 31:0 appended
2025-11-28T18:22:31.983203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 31.1/3
2025-11-28T18:22:31.983218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 1
2025-11-28T18:22:31.983221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 31:1 appended
2025-11-28T18:22:31.983237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 31.2/3
2025-11-28T18:22:31.983249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 2
2025-11-28T18:22:31.983440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 31 partNo 2 FormedBlobsCount 0 NewHead: Offset 30 PartNo 2 PackedSize 1073491 count 2 nextOffset 32 batches 3
2025-11-28T18:22:31.983458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 31:2 appended
2025-11-28T18:22:31.983518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 30,2 HeadOffset 23 endOffset 24 curOffset 32 d0000000000_00000000000000000030_00002_0000000002_00002| size 1073481 WTime 1764354151983
2025-11-28T18:22:31.983714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558396:3525] destroyed
2025-11-28T18:22:31.983731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843378610558401:3528], now have 1 active actors on pipe
2025-11-28T18:22:31.983762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.983769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.983788Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.983806Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 23 partNo 0 count 7 size 8365364
2025-11-28T18:22:31.983818Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 30 partNo 2 count 2 size 1073481
2025-11-28T18:22:31.983907Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 4 Topic 'rt3.dc1--topic1' partition 0 user user offset 0 partno 0 count 16 size 26214400 endOffset 32 max time lag 0ms effective offset 0
2025-11-28T18:22:31.984148Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 4 added 3 blobs, size 25145475 count 24 last offset 15, current partition end offset: 32
2025-11-28T18:22:31.984167Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 4. Send blob request.
2025-11-28T18:22:31.984180Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 7 parts_count 16 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:31.984188Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 7 partno 2 count 8 parts_count 15 source 1 size 7878063 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.984193Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 15 partno 1 count 8 parts_count 15 source 1 size 7878063 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.984208Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 4. Have to read 1 of 3 from KV
2025-11-28T18:22:31.984347Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 7 partno 2 count 8 parts 15 suffix '0'
2025-11-28T18:22:31.984356Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 15 partno 1 count 8 parts 15 suffix '0'
2025-11-28T18:22:31.984365Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 7 parts_count 16
2025-11-28T18:22:32.004944Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 3 from KV. Status 1
2025-11-28T18:22:32.004965Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-11-28T18:22:32.004975Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 7 size 8365339 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:32.005044Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 23 count 7 size 8365364 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:32.005055Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 2 suffix '124' size 1048780
2025-11-28T18:22:32.005058Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 30 count 2 size 1073481 actorID [2:7577843380661645505:2363]
2025-11-28T18:22:32.005061Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 7 parts 16 suffix '0' size 8365339
2025-11-28T18:22:32.005073Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:32.005077Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:32.005079Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 3 blobs
2025-11-28T18:22:32.005080Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:32.005083Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:32.005087Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:32.005091Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:32.005095Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:32.005098Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 7 parts 16 suffix '0' size 8365364
2025-11-28T18:22:32.005103Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-28T18:22:32.005136Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 7 partno 2 count 8 parts 15 suffix '0' size 7878063
2025-11-28T18:22:32.005145Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 30 partno 2 count 2 parts 2 suffix '124' size 1073481
2025-11-28T18:22:32.005272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.005429Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.005651Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-28T18:22:32.006163Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 1 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.006326Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 1 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-28T18:22:32.006487Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 2 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.006984Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 2 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-28T18:22:32.007130Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 3 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.007286Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 3 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-28T18:22:32.007433Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 4 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.007591Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 4 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-28T18:22:32.007741Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 5 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.007906Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 5 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-28T18:22:32.008052Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 6 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.008079Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 6 totakecount 7 count 1 size 24704 from pos 0 cbcount 1
2025-11-28T18:22:32.008709Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 7 totakecount 7 count 0 size 1024007 from pos 0 cbcount 2
2025-11-28T18:22:32.008776Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 7 totakecount 8 count 1 size 24704 from pos 0 cbcount 1
2025-11-28T18:22:32.008916Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 8 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.009065Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 8 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.009591Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 8 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-28T18:22:32.009755Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 9 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.009940Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 9 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-28T18:22:32.010100Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 10 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.010635Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 10 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-28T18:22:32.011143Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 11 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.011310Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 11 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-28T18:22:32.011833Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 12 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.012002Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 12 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-28T18:22:32.012155Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 13 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.012314Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 13 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-28T18:22:32.012476Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 14 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.012499Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 14 totakecount 8 count 1 size 24704 from pos 0 cbcount 1
2025-11-28T18:22:32.012645Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 15 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.012843Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 15 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.012866Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 15 totakecount 8 count 1 size 24704 from pos 0 cbcount 1
2025-11-28T18:22:32.012938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.013121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.013133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.013136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.013138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.013141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.013183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.013204Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 23 size 1048780 cause it's been evicted from L2. Actual L1 size: 13
2025-11-28T18:22:32.013212Z node 2 :PERSQUEUE NOTICE: read.h:372: [72075186224037892][PQCacheProxy]Have to remove new data from cache. Topic rt3.dc1--topic1, tablet id72075186224037892, cookie 0
2025-11-28T18:22:32.013222Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 29 size 1048780 cause it's been evicted from L2. Actual L1 size: 12
2025-11-28T18:22:32.013230Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 26 size 1048780 cause it's been evicted from L2. Actual L1 size: 11
2025-11-28T18:22:32.013233Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 30 size 1048780 cause it's been evicted from L2. Actual L1 size: 10
2025-11-28T18:22:32.013236Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 25 size 1048780 cause it's been evicted from L2. Actual L1 size: 9
2025-11-28T18:22:32.013241Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 27 size 1048780 cause it's been evicted from L2. Actual L1 size: 8
2025-11-28T18:22:32.013244Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 31 size 1048780 cause it's been evicted from L2. Actual L1 size: 7
2025-11-28T18:22:32.013247Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 7 size 7878063 cause it's been evicted from L2. Actual L1 size: 6
2025-11-28T18:22:32.013249Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 24 size 1048780 cause it's been evicted from L2. Actual L1 size: 5
2025-11-28T18:22:32.013251Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 28 size 1048780 cause it's been evicted from L2. Actual L1 size: 4
2025-11-28T18:22:32.013255Z node 2 :PERSQUEUE NOTICE: read.h:372: [72075186224037892][PQCacheProxy]Have to remove new data from cache. Topic rt3.dc1--topic1, tablet id72075186224037892, cookie 0
2025-11-28T18:22:32.013325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.013364Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.013379Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000024_00000_0000000001_00002?(+) to d0000000000_00000000000000000024_00000_0000000001_00002?(+)
2025-11-28T18:22:32.013381Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000025_00000_0000000001_00002?(+) to d0000000000_00000000000000000025_00000_0000000001_00002?(+)
2025-11-28T18:22:32.013383Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000026_00000_0000000001_00002?(+) to d0000000000_00000000000000000026_00000_0000000001_00002?(+)
2025-11-28T18:22:32.013386Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000027_00000_0000000001_00002?(+) to d0000000000_00000000000000000027_00000_0000000001_00002?(+)
2025-11-28T18:22:32.013387Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000028_00000_0000000001_00002?(+) to d0000000000_00000000000000000028_00000_0000000001_00002?(+)
2025-11-28T18:22:32.013389Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000029_00000_0000000001_00002?(+) to d0000000000_00000000000000000029_00000_0000000001_00002?(+)
2025-11-28T18:22:32.013392Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000030_00000_0000000001_00002?(+) to d0000000000_00000000000000000030_00000_0000000001_00002?(+)
2025-11-28T18:22:32.013394Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000031_00000_0000000001_00002?(+) to d0000000000_00000000000000000031_00000_0000000001_00002?(+)
2025-11-28T18:22:32.013396Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000023_00000_0000000001_00002|(+) to d0000000000_00000000000000000023_00000_0000000001_00002|(+)
2025-11-28T18:22:32.028527Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-28T18:22:32.031312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.031344Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.031354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.031358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.031360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.031363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.031365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.031374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.036061Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.045127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843378610558401:3528] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdReadResult {
    MaxOffset: 32
    Result {
      Offset: 0
      Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
>>>>> 2
CallPersQueueGRPC request to localhost:2670
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdRead {
    ClientId: "user"
    Offset: 16
    Count: 16
    ReadTimestampMs: 0
  }
}
2025-11-28T18:22:32.106940Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.107487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843382905525712:3539], now have 1 active actors on pipe
2025-11-28T18:22:32.107611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.107625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.107678Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 5 Topic 'rt3.dc1--topic1' partition 0 user user offset 16 partno 0 count 16 size 26214400 endOffset 32 max time lag 0ms effective offset 16
2025-11-28T18:22:32.108019Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 5 added 2 blobs, size 9438782 count 16 last offset 23, current partition end offset: 32
2025-11-28T18:22:32.108034Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 5. Send blob request.
2025-11-28T18:22:32.108051Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 15 partno 1 count 8 parts_count 15 source 1 size 7878063 accessed 1 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:32.108055Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 23 partno 0 count 7 parts_count 16 source 1 size 8365364 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.108068Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 5. All 2 blobs are from cache.
2025-11-28T18:22:32.108111Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 2 blobs
2025-11-28T18:22:32.108496Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 16 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.108644Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 16 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.108826Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 16 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-28T18:22:32.109109Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 17 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.109278Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 17 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-28T18:22:32.109662Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 18 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.109837Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 18 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-28T18:22:32.109983Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 19 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.110152Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 19 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-28T18:22:32.110406Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 20 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.110592Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 20 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-28T18:22:32.110867Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 21 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.111031Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 21 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-28T18:22:32.111185Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 22 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.111204Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 22 totakecount 8 count 1 size 24704 from pos 0 cbcount 1
2025-11-28T18:22:32.111422Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 23 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.111549Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 23 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.111565Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 23 totakecount 7 count 1 size 24704 from pos 0 cbcount 1
2025-11-28T18:22:32.111699Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 24 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.111838Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 24 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.111988Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 24 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-28T18:22:32.112121Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 25 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.112283Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 25 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-28T18:22:32.112468Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 26 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.112648Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 26 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-28T18:22:32.112812Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 27 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.112978Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 27 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-28T18:22:32.114090Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 28 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.114291Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 28 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-28T18:22:32.114690Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 15 partno 1 count 8 parts 15 suffix '0'
2025-11-28T18:22:32.114707Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 7 parts 16 suffix '0'
2025-11-28T18:22:32.115927Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 29 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-28T18:22:32.115972Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 29 totakecount 7 count 1 size 24704 from pos 0 cbcount 1
2025-11-28T18:22:32.116272Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 30 totakecount 7 count 0 size 1024007 from pos 0 cbcount 2
2025-11-28T18:22:32.116389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.129046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.129066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.129069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.129073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.129076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.134578Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.142137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843382905525712:3539] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdReadResult {
    MaxOffset: 32
    Result {
      Offset: 16
      Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-28T18:22:32.229193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.229213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.229216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.229219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.229221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.329574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.329592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.329594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.329597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.329599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
[[bad]]assertion failed at ydb/services/persqueue_v1/persqueue_ut.cpp:3269, virtual void NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TTestCaseCache::Execute_(NUnitTest::TTestContext &): (info0.BlobsFromCache == 1) failed: (2 != 1) [[rst]]
[[alt1]]TBackTrace::Capture()+28 (0x15100E9C)
NUnitTest::NPrivate::RaiseError(char const*, TBasicString<char, std::__y1::char_traits<char>> const&, bool)+128 (0x152BCEA0)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TTestCaseCache::Execute_(NUnitTest::TTestContext&)+4375 (0x14E2B1B7)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TCurrentTest::Execute()::'lambda'()::operator()() const+71 (0x14F17D07)
NUnitTest::TTestBase::Run(std::__y1::function<void ()>, TBasicString<char, std::__y1::char_traits<char>> const&, char const*, bool)+120 (0x152BF078)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TCurrentTest::Execute()+436 (0x14F17604)
NUnitTest::TTestFactory::Execute()+817 (0x152BF7C1)
NUnitTest::RunMain(int, char**)+3229 (0x152D1C2D)
??+0 (0x7F6D6B9BFD90)
__libc_start_main+128 (0x7F6D6B9BFE40)
_start+41 (0x13F1F029)
[[rst]]
2025-11-28T18:22:25.689526Z node 1 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[1:7577843355284271681:2262];send_to=[0:7307199536658146131:7762515];
2025-11-28T18:22:25.689590Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
2025-11-28T18:22:25.692031Z node 1 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-11-28T18:22:25.692581Z node 2 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[2:7577843354253914955:2158];send_to=[0:7307199536658146131:7762515];
2025-11-28T18:22:25.692674Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
test_client.cpp: SetPath # /home/runner/.ya/build/build_root/1xri/0001f7/r3tmp/tmpgIYkrc/pdisk_1.dat
2025-11-28T18:22:25.694778Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-11-28T18:22:25.726651Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-11-28T18:22:25.738276Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-11-28T18:22:25.740886Z node 1 :IMPORT WARN: schemeshard_import.cpp:423: Table profiles were not loaded
TServer::EnableGrpc on GrpcPort 19639, node 1
2025-11-28T18:22:25.748232Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:190: distributable config is empty, broken or outdated, will use file: /home/runner/.ya/build/build_root/1xri/0001f7/r3tmp/yandexNndF3j.tmp
2025-11-28T18:22:25.748254Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:196: will try to initialize from file: /home/runner/.ya/build/build_root/1xri/0001f7/r3tmp/yandexNndF3j.tmp
2025-11-28T18:22:25.748311Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:202: successfully initialized from file: /home/runner/.ya/build/build_root/1xri/0001f7/r3tmp/yandexNndF3j.tmp
2025-11-28T18:22:25.748331Z node 1 :NET_CLASSIFIER ERROR: net_classifier.cpp:228: got bad distributable configuration
2025-11-28T18:22:25.752179Z INFO: TTestServer started on Port 1534 GrpcPort 19639
TClient is connected to server localhost:1534
PQClient connected to localhost:19639
WaitRootIsUp 'Root'...
TClient::Ls request: Root
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "Root"
    PathId: 1
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 1
    CreateStep: 0
    ParentPathId: 1
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 1
      SubDomainVersion: 0
      SecurityStateVersion: 0
    }
    ChildrenExist: false
  }
  Children {
    Name: ".sys"
    PathId: 18446744073709551615
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 0
    CreateStep: 0
    ParentPathId: 18446744073709551615
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 0
      Pl...
(TRUNCATED)
WaitRootIsUp 'Root' success.
2025-11-28T18:22:25.789065Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-11-28T18:22:25.789100Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-11-28T18:22:25.790542Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-11-28T18:22:25.796696Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpAlterSubDomain, opId: 281474976720657:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_alter_subdomain.cpp:311)
waiting...
2025-11-28T18:22:25.799461Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-11-28T18:22:25.799489Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-11-28T18:22:25.800386Z node 1 :HIVE WARN: hive_impl.cpp:811: HIVE#72057594037968897 Handle TEvInterconnect::TEvNodeConnected, NodeId 2 Cookie 2
2025-11-28T18:22:25.800656Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-11-28T18:22:25.802350Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__notify.cpp:122: NotifyTxCompletion, unknown transaction, txId: 281474976720657, at schemeshard: 72057594046644480
waiting...
2025-11-28T18:22:25.808839Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720659:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
waiting...
waiting...
waiting...
2025-11-28T18:22:26.020163Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-11-28T18:22:26.048904Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7577843358548882515:2300], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-28T18:22:26.048928Z node 2 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-28T18:22:26.048929Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7577843358548882536:2303], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-28T18:22:26.049000Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7577843358548882543:2305], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-28T18:22:26.049009Z node 2 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-28T18:22:26.050028Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateResourcePool, opId: 281474976715657:3, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:179)
2025-11-28T18:22:26.055174Z node 2 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [2:7577843358548882542:2304], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976715657 completed, doublechecking }
2025-11-28T18:22:26.127062Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720662:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
2025-11-28T18:22:26.127539Z node 1 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:625: Compilation failed, self: [1:7577843359579239836:2331], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-11-28T18:22:26.127704Z node 1 :KQP_SESSION WARN: kqp_session_actor.cpp:2542: SessionId: ydb://session/3?node_id=1&id=NmVmYjFhNjUtMzBhMTZlNWUtNGMxNTQzNTItYWFkNDNkZQ==, ActorId: [1:7577843359579239794:2323], ActorState: ExecuteState, TraceId: 01kb5v7fsb885fs48w1114j5nz, ReplyQueryCompileError, status: SCHEME_ERROR, issues: { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } }, remove tx with tx_id: 
2025-11-28T18:22:26.128098Z node 1 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-11-28T18:22:26.136849Z node 2 :TX_PROXY ERROR: schemereq.cpp:593: Actor# [2:7577843358548882571:2172] txid# 281474976715658, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 9], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-11-28T18:22:26.141097Z node 2 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:625: Compilation failed, self: [2:7577843358548882585:2309], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-11-28T18:22:26.141593Z node 2 :KQP_SESSION WARN: kqp_session_actor.cpp:2542: SessionId: ydb://session/3?node_id=2&id=MzcyZWYxZGEtNDQwNTk0NzYtOWNlN2JmN2MtNzhhMzJhYWU=, ActorId: [2:7577843358548882510:2298], ActorState: ExecuteState, TraceId: 01kb5v7fr0e4nfr14a3wxpz353, ReplyQueryCompileError, status: SCHEME_ERROR, issues: { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } }, remove tx with tx_id: 
2025-11-28T18:22:26.141697Z node 2 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-11-28T18:22:26.143014Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720663:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
2025-11-28T18:22:26.163554Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720664:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
=== Init DC: UPSERT INTO `/Root/PQ/Config/V2/Cluster` (name, balancer, local, enabled, weight) VALUES ("dc1", "localhost", true, true, 1000), ("dc2", "dc2.logbroker.yandex.net", false, true, 1000);
=== CheckClustersList. Subcribe to ClusterTracker from [1:7577843359579240266:3031] 
2025-11-28T18:22:26.689869Z node 1 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-11-28T18:22:26.694483Z node 2 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-11-28T18:22:30.689574Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[1:7577843355284271681:2262];send_to=[0:7307199536658146131:7762515];
2025-11-28T18:22:30.689620Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
2025-11-28T18:22:30.692668Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[2:7577843354253914955:2158];send_to=[0:7307199536658146131:7762515];
2025-11-28T18:22:30.692709Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
=== CheckClustersList. Ok
PQ Client: create topic: rt3.dc1--topic1 with 1 partitions
CallPersQueueGRPC request to localhost:19639
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--topic1"
  }
}
2025-11-28T18:22:31.410803Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--topic1, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:19639
MetaRequest {
  CmdCreateTopic {
    Topic: "rt3.dc1--topic1"
    NumPartitions: 1
    Config {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 6291456
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
  }
}
CallPersQueueGRPC response:
Status: 129
ProxyErrorCode: 53
SchemeStatus: 1
FlatTxId {
  TxId: 281474976720671
  SchemeShardTabletId: 72057594046644480
  PathId: 13
}
ErrorCode: OK
AddTopic: rt3.dc1--topic1
===Run query:``DECLARE $version as Int64; DECLARE $path AS Utf8; DECLARE $cluster as Utf8; UPSERT INTO `/Root/PQ/Config/V2/Topics` (path, dc) VALUES ($path, $cluster); UPSERT INTO `/Root/PQ/Config/V2/Versions` (name, version) VALUES ("Topics", $version);`` with topic = topic1, dc = dc1
2025-11-28T18:22:31.420481Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1656: [72075186224037893][] pipe [1:7577843381054077169:3264] connected; active server actors: 1
2025-11-28T18:22:31.420623Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1520: [72075186224037893][rt3.dc1--topic1] updating configuration. Deleted partitions []. Added partitions [0]
2025-11-28T18:22:31.420909Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:942: [72075186224037893][rt3.dc1--topic1] Discovered subdomain [OwnerId: 72057594046644480, LocalPathId: 1] state, outOfSpace = 0 at RB 72075186224037893
2025-11-28T18:22:31.421348Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:140: [72075186224037893][rt3.dc1--topic1] BALANCER INIT DONE for rt3.dc1--topic1: (0, 72075186224037892) 
2025-11-28T18:22:31.427723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2959: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-11-28T18:22:31.433020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2991: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-11-28T18:22:31.433116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:787: [PQ: 72075186224037892] no config, start with empty partitions and default config
2025-11-28T18:22:31.433125Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:913: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-11-28T18:22:31.433140Z node 2 :PERSQUEUE INFO: pq_impl.cpp:612: [PQ: 72075186224037892] doesn't have tx writes info
2025-11-28T18:22:31.433277Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077168:3263], now have 1 active actors on pipe
2025-11-28T18:22:31.433288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077193:2450], now have 1 active actors on pipe
2025-11-28T18:22:31.433606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3563: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-28T18:22:31.434226Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:404: [72075186224037893][rt3.dc1--topic1] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1
2025-11-28T18:22:31.434756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-28T18:22:31.441300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077231:3311], now have 1 active actors on pipe
2025-11-28T18:22:31.441652Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:84: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitConfigStep
2025-11-28T18:22:31.441682Z node 2 :PERSQUEUE DEBUG: partition.cpp:1315: [72075186224037892][Partition][0][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-11-28T18:22:31.441788Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:84: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-11-28T18:22:31.441895Z node 2 :PERSQUEUE INFO: partition_init.cpp:1177: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7577843380023719328:2363]
2025-11-28T18:22:31.442607Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:84: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitFieldsStep
2025-11-28T18:22:31.442621Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:64: [rt3.dc1--topic1:0:Initializer] Initializing completed.
2025-11-28T18:22:31.442627Z node 2 :PERSQUEUE INFO: partition.cpp:707: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--topic1' partition 0 generation 1 [2:7577843380023719328:2363]
2025-11-28T18:22:31.442634Z node 2 :PERSQUEUE DEBUG: partition.cpp:727: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--topic1 partitition 0 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-11-28T18:22:31.442642Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:107: [72075186224037892][Partition][0][StateIdle] Initializing MLP Consumers: 0
2025-11-28T18:22:31.442944Z node 2 :PERSQUEUE DEBUG: partition.cpp:4408: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 1
2025-11-28T18:22:31.442962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.442979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2392: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TTransaction[ProposeConfig])
2025-11-28T18:22:31.442997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.443005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.443008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.443010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.443029Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.443037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.443039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.443040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.443042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.443050Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:930: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-11-28T18:22:31.443058Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:70: [72075186224037892][Partition][0][StateIdle] Process MLP pending events. Count 0
2025-11-28T18:22:31.443146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3563: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-28T18:22:31.443197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.445043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-28T18:22:31.445087Z node 2 :PERSQUEUE DEBUG: partition.cpp:1420: [72075186224037892][Partition][0][StateIdle] Handle TEvPQ::TEvTxCommit Step 1764354151490, TxId 281474976720671
2025-11-28T18:22:31.445092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.445096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.445098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.445104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2456: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TTransaction[ProposeConfig])
2025-11-28T18:22:31.445122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.445132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.445134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.445182Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.445944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.445985Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:107: [72075186224037892][Partition][0][StateIdle] Initializing MLP Consumers: 0
2025-11-28T18:22:31.446020Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.446032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.446035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.446038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.446040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.446042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.446048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.446139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:398: [PQ: 72075186224037892] Apply new config PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 6291456 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--topic1" Version: 0 LocalDC: true Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } TopicPath: "/Root/PQ/rt3.dc1--topic1" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Version: 0 Generation: 0 } MonitoringProjectId: ""
2025-11-28T18:22:31.446159Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:913: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-11-28T18:22:31.446207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3563: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-28T18:22:31.447250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-28T18:22:31.447287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3563: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-28T18:22:31.447979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
===Query complete
TClient::Ls request: /Root/PQ/rt3.dc1--topic1
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--topic1"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720671
    CreateStep: 1764354151490
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--topic1"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 6291456
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 600000...
(TRUNCATED)
GetTopicVersionFromPath:  record Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--topic1"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720671
    CreateStep: 1764354151490
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--topic1"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 6291456
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      YdbDatabasePath: "/Root"
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
    Partitions {
      PartitionId: 0
      TabletId: 72075186224037892
      Status: Active
    }
    AlterVersion: 1
    BalancerTabletID: 72075186224037893
    NextPartitionId: 1
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 1
      PlanResolution: 7
      Coordinators: 72057594046316545
      TimeCastBucketsPerMediator: 2
      Mediators: 72057594046382081
    }
    DomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    PathsInside: 12
    PathsLimit: 10000
    ShardsInside: 6
    ShardsLimit: 200000
    ResourcesDomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    DiskSpaceUsage {
      Tables {
        TotalSize: 0
        DataSize: 0
        IndexSize: 0
      }
      Topics {
        ReserveSize: 0
        AccountSize: 0
        DataSize: 0
        UsedReserveSize: 0
      }
    }
    PQPartitionsInside: 1
    PQPartitionsLimit: 1000000
    SchemeLimits {
      MaxDepth: 32
      MaxPaths: 10000
      MaxChildrenInDir: 100000
      MaxAclBytesSize: 10240
      MaxTableColumns: 200
      MaxTableColumnNameLength: 255
      MaxTableKeyColumns: 30
      MaxTableIndices: 20
      MaxShards: 200000
      MaxShardsInPath: 35000
      MaxConsistentCopyTargets: 10000
      MaxPathElementLength: 255
      ExtraPathSymbolsAllowed: "!\"#$%&\'()*+,-.:;<=>?@[\\]^_`{|}~"
      MaxPQPartitions: 1000000
      MaxTableCdcStreams: 5
      MaxExports: 10
      MaxImports: 10
      MaxColumnTableColumns: 10000
    }
  }
}
Path: "/Root/PQ/rt3.dc1--topic1"
 name rt3.dc1--topic1 version1
CallPersQueueGRPC request to localhost:19639
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--topic1"
  }
}
2025-11-28T18:22:31.459368Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
MetaResponse {
  CmdGetTopicMetadataResult {
    TopicInfo {
      Topic: "rt3.dc1--topic1"
      NumPartitions: 1
      Config {
        PartitionConfig {
          LifetimeSeconds: 86400
          LowWatermark: 6291456
          SourceIdLifetimeSeconds: 86400
          WriteSpeedInBytesPerSecond: 20000000
          BurstSize: 20000000
          SourceIdMaxCounts: 6000000
        }
        Version: 1
        LocalDC: true
        Codecs {
          Ids: 0
          Ids: 1
          Ids: 2
          Codecs: "raw"
          Codecs: "gzip"
          Codecs: "lzop"
        }
        TopicPath: "/Root/PQ/rt3.dc1--topic1"
        YdbDatabasePath: "/Root"
        Consumers {
          Name: "user"
          ReadFromTimestampsMs: 0
          FormatVersion: 0
          Version: 0
        }
      }
      ErrorCode: OK
    }
  }
}
=== Topic created, have version: 1
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.460471Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.462510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077286:3344], now have 1 active actors on pipe
2025-11-28T18:22:31.462541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.462549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.462592Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b1a50637-c2230437-c2b11c55-f8c38314_0 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.462608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.462621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.462628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.462631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.462635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.462645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.462652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.462656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.462674Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.462697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.462830Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b1a50637-c2230437-c2b11c55-f8c38314_0"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b1a50637-c2230437-c2b11c55-f8c38314_0"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 0
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.464287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077286:3344] destroyed
2025-11-28T18:22:31.466611Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.467318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077289:3345], now have 1 active actors on pipe
2025-11-28T18:22:31.468429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.468445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.468533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 0 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.468544Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.468555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.468558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.468567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.468569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.468573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.469851Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 0
2025-11-28T18:22:31.469897Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 0 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 262214 count 1 nextOffset 1 batches 1
2025-11-28T18:22:31.469905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.469910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.469913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.469993Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 0,1 HeadOffset 0 endOffset 0 curOffset 1 d0000000000_00000000000000000000_00000_0000000001_00000? size 262204 WTime 1764354151469
2025-11-28T18:22:31.470035Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.470045Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 1 size 262204
2025-11-28T18:22:31.472451Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 0 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.472472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.472493Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.472492Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.472500Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.472508Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 0, partNo: 0, Offset: 0 is stored on disk
2025-11-28T18:22:31.472538Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:930: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-11-28T18:22:31.472542Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:972: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-11-28T18:22:31.472545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.472547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.472549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.472551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.472552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.472557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.472569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.472603Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--topic1' partition 0 user user offset 0 partno 0 count 1 size 1024000 endOffset 1 max time lag 0ms effective offset 0
2025-11-28T18:22:31.472617Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 262204 count 1 last offset 0, current partition end offset: 1
2025-11-28T18:22:31.472619Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-11-28T18:22:31.472628Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.472638Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 3. All 1 blobs are from cache.
2025-11-28T18:22:31.472649Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.472650Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 1 blobs
2025-11-28T18:22:31.472767Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 1 count 1 size 262182 from pos 0 cbcount 1
2025-11-28T18:22:31.472785Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1022: Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp done, result 1764354151469 queuesize 0 startOffset 0
2025-11-28T18:22:31.472996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077289:3345] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 0
    Offset: 0
    WriteTimestampMS: 1764354151468
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:31.472655Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.473796Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.474225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077293:3347], now have 1 active actors on pipe
2025-11-28T18:22:31.474310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.474316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.474344Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ac707d08-fdb86e67-16498b94-ce5043c2_1 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.474352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.474355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.474359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.474361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.474365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.474372Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.474375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.474378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.474389Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.474397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.474538Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.474809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077293:3347] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ac707d08-fdb86e67-16498b94-ce5043c2_1"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ac707d08-fdb86e67-16498b94-ce5043c2_1"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 1
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.476866Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.477439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077296:3348], now have 1 active actors on pipe
2025-11-28T18:22:31.477883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.477895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.477915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 1 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.477927Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.477939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.477943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.477953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.477955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.477958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.479040Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 0
2025-11-28T18:22:31.479086Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 1 partNo 0 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 262214 count 1 nextOffset 2 batches 1
2025-11-28T18:22:31.479095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.479098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.479103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.479202Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 1,1 HeadOffset 1 endOffset 1 curOffset 2 d0000000000_00000000000000000001_00000_0000000001_00000? size 262204 WTime 1764354151479
2025-11-28T18:22:31.479248Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.479259Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 1 partNo 0 count 1 size 262204
2025-11-28T18:22:31.482329Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 1 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.482348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.482369Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.482378Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.482388Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 1, partNo: 0, Offset: 1 is stored on disk
2025-11-28T18:22:31.482421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.482425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.482427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.482430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.482432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.482439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.482452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.482528Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.482599Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.482984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077296:3348] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 1
    Offset: 1
    WriteTimestampMS: 1764354151477
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.484490Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.485009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077300:3350], now have 1 active actors on pipe
2025-11-28T18:22:31.485086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.485098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.485130Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a36e8651-816d3d01-cd71e51-7093ae7f_2 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.485148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.485151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.485155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.485158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.485162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.485170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.485172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.485176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.485189Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.485209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.485281Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a36e8651-816d3d01-cd71e51-7093ae7f_2"
    Status: Active
    SeqNo: 0
  }
}
2025-11-28T18:22:31.485557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077300:3350] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a36e8651-816d3d01-cd71e51-7093ae7f_2"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 2
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.487897Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.488616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077303:3351], now have 1 active actors on pipe
2025-11-28T18:22:31.488839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.488853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.488871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 2 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.488882Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.488893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.488902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.488911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.488918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.488922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.490095Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 0
2025-11-28T18:22:31.490143Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 2 partNo 0 FormedBlobsCount 0 NewHead: Offset 2 PartNo 0 PackedSize 262214 count 1 nextOffset 3 batches 1
2025-11-28T18:22:31.490158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.490161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.490164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.490264Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 2,1 HeadOffset 2 endOffset 2 curOffset 3 d0000000000_00000000000000000002_00000_0000000001_00000? size 262204 WTime 1764354151489
2025-11-28T18:22:31.490318Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.490335Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 2 partNo 0 count 1 size 262204
2025-11-28T18:22:31.492631Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 2 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.492657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.492677Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.492679Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.492692Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.492701Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 2, partNo: 0, Offset: 2 is stored on disk
2025-11-28T18:22:31.492746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.492754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.492755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.492758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.492760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.492768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.492780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.492881Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 2
    Offset: 2
    WriteTimestampMS: 1764354151488
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:31.493323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077303:3351] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.493949Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.494367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077307:3353], now have 1 active actors on pipe
2025-11-28T18:22:31.494528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.494538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.494575Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|bbfa6903-8f55a712-744fd4bf-4c879f35_3 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.494587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.494591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.494595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.494602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.494605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.494615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.494625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.494628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.494640Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.494657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.494728Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|bbfa6903-8f55a712-744fd4bf-4c879f35_3"
    Status: Active
    SeqNo: 0
  }
}
2025-11-28T18:22:31.495090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077307:3353] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|bbfa6903-8f55a712-744fd4bf-4c879f35_3"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 3
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.497398Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.497769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077310:3354], now have 1 active actors on pipe
2025-11-28T18:22:31.498210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.498226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.498244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 3 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.498255Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.498267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.498276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.498286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.498289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.498293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.499354Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 0
2025-11-28T18:22:31.499396Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 3 partNo 0 FormedBlobsCount 0 NewHead: Offset 3 PartNo 0 PackedSize 262214 count 1 nextOffset 4 batches 1
2025-11-28T18:22:31.499404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.499409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.499412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.499501Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 3,1 HeadOffset 3 endOffset 3 curOffset 4 d0000000000_00000000000000000003_00000_0000000001_00000? size 262204 WTime 1764354151498
2025-11-28T18:22:31.499548Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.499564Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 3 partNo 0 count 1 size 262204
2025-11-28T18:22:31.501967Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 3 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.501998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.502021Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.502023Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.502031Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.502041Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 3, partNo: 0, Offset: 3 is stored on disk
2025-11-28T18:22:31.502079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.502087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.502090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.502092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.502095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.502103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.502116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.502201Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.502611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077310:3354] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 3
    Offset: 3
    WriteTimestampMS: 1764354151497
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.503367Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.503766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077314:3356], now have 1 active actors on pipe
2025-11-28T18:22:31.503800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.503810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.503845Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a13919cc-92387f66-d0e4a739-6e7190f2_4 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.503859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.503862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.503867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.503868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.503872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.503881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.503888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.503890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.503903Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.503918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.503973Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.504193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077314:3356] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a13919cc-92387f66-d0e4a739-6e7190f2_4"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a13919cc-92387f66-d0e4a739-6e7190f2_4"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 4
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.506426Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.509600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077317:3357], now have 1 active actors on pipe
2025-11-28T18:22:31.510024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.510042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.510071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 4 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.510085Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.510105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.510114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.510127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.510129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.510133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.511188Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 0
2025-11-28T18:22:31.511236Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 4 partNo 0 FormedBlobsCount 0 NewHead: Offset 4 PartNo 0 PackedSize 262214 count 1 nextOffset 5 batches 1
2025-11-28T18:22:31.511252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.511256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.511260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.511374Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 4,1 HeadOffset 4 endOffset 4 curOffset 5 d0000000000_00000000000000000004_00000_0000000001_00000? size 262204 WTime 1764354151511
2025-11-28T18:22:31.511432Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.511450Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 4 partNo 0 count 1 size 262204
2025-11-28T18:22:31.514530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 4 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.514779Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.514566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.514592Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.514594Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.514603Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.514617Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 4, partNo: 0, Offset: 4 is stored on disk
2025-11-28T18:22:31.514659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.514662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.514663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.514666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.514668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.514677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.514690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 4
    Offset: 4
    WriteTimestampMS: 1764354151509
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.515397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077317:3357] destroyed
2025-11-28T18:22:31.515667Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.516145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077321:3359], now have 1 active actors on pipe
2025-11-28T18:22:31.516226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.516234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.516271Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9d47bce3-44986b08-eb57d1e6-74f1a3f9_5 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.516279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.516283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.516288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.516290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.516293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.516304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.516307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.516310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.516377Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.516321Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.516332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.516567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077321:3359] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9d47bce3-44986b08-eb57d1e6-74f1a3f9_5"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9d47bce3-44986b08-eb57d1e6-74f1a3f9_5"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 5
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.518796Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.519210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077324:3360], now have 1 active actors on pipe
2025-11-28T18:22:31.519605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.519616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.519637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 5 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.519647Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.519660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.519663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.519672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.519674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.519679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.520820Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 0
2025-11-28T18:22:31.520855Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 5 partNo 0 FormedBlobsCount 0 NewHead: Offset 5 PartNo 0 PackedSize 262214 count 1 nextOffset 6 batches 1
2025-11-28T18:22:31.520864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.520866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.520869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.520972Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 5,1 HeadOffset 5 endOffset 5 curOffset 6 d0000000000_00000000000000000005_00000_0000000001_00000? size 262204 WTime 1764354151520
2025-11-28T18:22:31.521016Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.521029Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 5 partNo 0 count 1 size 262204
2025-11-28T18:22:31.523362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 5 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.523386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.523405Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.523409Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.523414Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.523423Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 5, partNo: 0, Offset: 5 is stored on disk
2025-11-28T18:22:31.523453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.523456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.523458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.523460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.523462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.523469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.523480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.523546Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 5
    Offset: 5
    WriteTimestampMS: 1764354151519
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.524128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077324:3360] destroyed
2025-11-28T18:22:31.524337Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.524707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077328:3362], now have 1 active actors on pipe
2025-11-28T18:22:31.524827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.524834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.524868Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|91cd7788-6959dc72-3b83ecd0-2f12b53f_6 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.524875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.524878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.524882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.524884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.524887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.524896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.524898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.524901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.524912Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.524922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.524966Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|91cd7788-6959dc72-3b83ecd0-2f12b53f_6"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|91cd7788-6959dc72-3b83ecd0-2f12b53f_6"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 6
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.525973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077328:3362] destroyed
2025-11-28T18:22:31.527284Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.528061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077331:3363], now have 1 active actors on pipe
2025-11-28T18:22:31.528220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.528239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.528263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 6 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.528282Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.528301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.528311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.528322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.528330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.528335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.529525Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 0
2025-11-28T18:22:31.529572Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 6 partNo 0 FormedBlobsCount 0 NewHead: Offset 6 PartNo 0 PackedSize 262214 count 1 nextOffset 7 batches 1
2025-11-28T18:22:31.529588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.529591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.529596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.529703Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 6,1 HeadOffset 6 endOffset 6 curOffset 7 d0000000000_00000000000000000006_00000_0000000001_00000? size 262204 WTime 1764354151529
2025-11-28T18:22:31.529765Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.529783Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 6 partNo 0 count 1 size 262204
2025-11-28T18:22:31.532334Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 6 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.532363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.532383Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.532386Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.532400Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.532417Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 6, partNo: 0, Offset: 6 is stored on disk
2025-11-28T18:22:31.532454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.532464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.532466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.532469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.532471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.532482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.532496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.532570Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.532899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077331:3363] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 6
    Offset: 6
    WriteTimestampMS: 1764354151527
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.533552Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.534077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077335:3365], now have 1 active actors on pipe
2025-11-28T18:22:31.534103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.534112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.534149Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|20ca110a-bf695a33-9fef85bb-ae987186_7 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.534159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.534162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.534167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.534171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.534175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.534185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.534189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.534263Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.534191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.534204Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.534215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
2025-11-28T18:22:31.534448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077335:3365] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|20ca110a-bf695a33-9fef85bb-ae987186_7"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|20ca110a-bf695a33-9fef85bb-ae987186_7"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 7
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.536503Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.536936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077338:3366], now have 1 active actors on pipe
2025-11-28T18:22:31.537327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.537340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.537362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 7 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.537371Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.537383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.537387Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.537396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.537399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.537403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.538393Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 0
2025-11-28T18:22:31.538431Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 7 partNo 0 FormedBlobsCount 0 NewHead: Offset 7 PartNo 0 PackedSize 262214 count 1 nextOffset 8 batches 1
2025-11-28T18:22:31.538441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.538444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.538448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.538628Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 7,1 HeadOffset 7 endOffset 7 curOffset 8 d0000000000_00000000000000000007_00000_0000000001_00000? size 262204 WTime 1764354151538
2025-11-28T18:22:31.538682Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.538696Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 7 partNo 0 count 1 size 262204
2025-11-28T18:22:31.541284Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 7 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.541316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.541339Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.541349Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.541353Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.541358Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 7, partNo: 0, Offset: 7 is stored on disk
2025-11-28T18:22:31.541394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.541404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.541406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.541409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.541411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.541421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.541434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.541580Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 7
    Offset: 7
    WriteTimestampMS: 1764354151537
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:31.541957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077338:3366] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.542670Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.543072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.543085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.543088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.543090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.543093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.543135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077342:3368], now have 1 active actors on pipe
2025-11-28T18:22:31.543285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.543293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.543329Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a56893d2-1641081b-2615fab6-9a345c15_8 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.543464Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.543339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.543342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.543347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.543349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.543353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.543362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.543367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.543369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.543378Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.543390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.543768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077342:3368] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a56893d2-1641081b-2615fab6-9a345c15_8"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a56893d2-1641081b-2615fab6-9a345c15_8"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 8
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.545938Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.546363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077345:3369], now have 1 active actors on pipe
2025-11-28T18:22:31.546851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.546876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.546898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 8 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.546911Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.546926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.546936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.546948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.546955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.546959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.547938Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 0
2025-11-28T18:22:31.547983Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 8 partNo 0 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 262214 count 1 nextOffset 9 batches 1
2025-11-28T18:22:31.547993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.547997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.548000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.548100Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 8,1 HeadOffset 8 endOffset 8 curOffset 9 d0000000000_00000000000000000008_00000_0000000001_00000? size 262204 WTime 1764354151547
2025-11-28T18:22:31.548156Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.548173Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 8 partNo 0 count 1 size 262204
2025-11-28T18:22:31.550535Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 8 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.550564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.550584Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.550594Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.550594Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.550607Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 8, partNo: 0, Offset: 8 is stored on disk
2025-11-28T18:22:31.550640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.550647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.550649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.550651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.550653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.550661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.550672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.550767Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 8
    Offset: 8
    WriteTimestampMS: 1764354151546
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:31.551207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077345:3369] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.551732Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.552129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077349:3371], now have 1 active actors on pipe
2025-11-28T18:22:31.552217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.552228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.552260Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|eef8e7a9-2527e03a-70815007-39869d84_9 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.552271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.552274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.552279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.552281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.552285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.552294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.552301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.552303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.552315Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.552326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.552389Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.552670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077349:3371] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|eef8e7a9-2527e03a-70815007-39869d84_9"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|eef8e7a9-2527e03a-70815007-39869d84_9"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 9
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.554113Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.554420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077352:3372], now have 1 active actors on pipe
2025-11-28T18:22:31.554855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.554871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.554893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 9 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.554904Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.554916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.554919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.554928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.554930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.554934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.555960Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 0
2025-11-28T18:22:31.555997Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 9 partNo 0 FormedBlobsCount 0 NewHead: Offset 9 PartNo 0 PackedSize 262214 count 1 nextOffset 10 batches 1
2025-11-28T18:22:31.556005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.556007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.556011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.556105Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 9,1 HeadOffset 9 endOffset 9 curOffset 10 d0000000000_00000000000000000009_00000_0000000001_00000? size 262204 WTime 1764354151555
2025-11-28T18:22:31.556149Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.556160Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 9 partNo 0 count 1 size 262204
2025-11-28T18:22:31.558892Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 9 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.558921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.558944Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.558956Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.558965Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.558974Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 9, partNo: 0, Offset: 9 is stored on disk
2025-11-28T18:22:31.559004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.559008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.559010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.559012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.559014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.559020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.559031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.559109Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.559423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077352:3372] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 9
    Offset: 9
    WriteTimestampMS: 1764354151554
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.560153Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.560546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077356:3374], now have 1 active actors on pipe
2025-11-28T18:22:31.560624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.560638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.560671Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|1911691b-86d0bf3e-ba0d38be-b69a7609_10 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.560683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.560686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.560691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.560696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.560699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.560709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.560715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.560718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.560741Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.560754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.560878Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.561128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077356:3374] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|1911691b-86d0bf3e-ba0d38be-b69a7609_10"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|1911691b-86d0bf3e-ba0d38be-b69a7609_10"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 10
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.563017Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.563331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077359:3375], now have 1 active actors on pipe
2025-11-28T18:22:31.563755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.563766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.563782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 10 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.563790Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.563798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.563804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.563812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.563814Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.563816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.564500Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 0
2025-11-28T18:22:31.564529Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 10 partNo 0 FormedBlobsCount 0 NewHead: Offset 10 PartNo 0 PackedSize 262214 count 1 nextOffset 11 batches 1
2025-11-28T18:22:31.564537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.564539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.564541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.564600Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 10,1 HeadOffset 10 endOffset 10 curOffset 11 d0000000000_00000000000000000010_00000_0000000001_00000? size 262204 WTime 1764354151564
2025-11-28T18:22:31.564637Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.564648Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 10 partNo 0 count 1 size 262204
2025-11-28T18:22:31.566697Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 10 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.566720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.566738Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.566740Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.566748Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.566758Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 10, partNo: 0, Offset: 10 is stored on disk
2025-11-28T18:22:31.566790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.566796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.566799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.566801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.566803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.566809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.566819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.566879Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.567192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077359:3375] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 10
    Offset: 10
    WriteTimestampMS: 1764354151563
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.567786Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.568127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077365:3379], now have 1 active actors on pipe
2025-11-28T18:22:31.568206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.568216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.568251Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|abae9eff-79d28bc-6d9aa166-e8a0b115_11 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.568263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.568267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.568272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.568275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.568278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.568286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.568292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.568296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.568306Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.568320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.568362Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.568538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077365:3379] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|abae9eff-79d28bc-6d9aa166-e8a0b115_11"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|abae9eff-79d28bc-6d9aa166-e8a0b115_11"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 11
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.570389Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.570750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077368:3380], now have 1 active actors on pipe
2025-11-28T18:22:31.571106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.571118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.571139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 11 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.571148Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.571164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.571168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.571176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.571182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.571186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.572627Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 0
2025-11-28T18:22:31.572683Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 11 partNo 0 FormedBlobsCount 0 NewHead: Offset 11 PartNo 0 PackedSize 262214 count 1 nextOffset 12 batches 1
2025-11-28T18:22:31.572696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.572699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.572702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.572822Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 11,1 HeadOffset 11 endOffset 11 curOffset 12 d0000000000_00000000000000000011_00000_0000000001_00000? size 262204 WTime 1764354151572
2025-11-28T18:22:31.572875Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.572894Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 11 partNo 0 count 1 size 262204
2025-11-28T18:22:31.574933Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 11 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.574956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.574976Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.574978Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.574986Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.574995Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 11, partNo: 0, Offset: 11 is stored on disk
2025-11-28T18:22:31.575027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.575034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.575036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.575038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.575040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.575047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.575057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.575125Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.575397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077368:3380] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 11
    Offset: 11
    WriteTimestampMS: 1764354151570
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.576214Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.576574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077372:3382], now have 1 active actors on pipe
2025-11-28T18:22:31.576620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.576628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.576659Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ba49d4ba-244083ea-2e26d428-5d8b0131_12 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.576669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.576671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.576676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.576679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.576682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.576690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.576701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.576703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.576713Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.576733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.576773Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.576940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077372:3382] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ba49d4ba-244083ea-2e26d428-5d8b0131_12"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ba49d4ba-244083ea-2e26d428-5d8b0131_12"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 12
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.579021Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.579357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077375:3383], now have 1 active actors on pipe
2025-11-28T18:22:31.579680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.579690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.579708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 12 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.579715Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.579727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.579735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.579745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.579751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.579754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.580463Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 0
2025-11-28T18:22:31.580497Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 12 partNo 0 FormedBlobsCount 0 NewHead: Offset 12 PartNo 0 PackedSize 262214 count 1 nextOffset 13 batches 1
2025-11-28T18:22:31.580505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.580512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.580514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.580600Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 12,1 HeadOffset 12 endOffset 12 curOffset 13 d0000000000_00000000000000000012_00000_0000000001_00000? size 262204 WTime 1764354151580
2025-11-28T18:22:31.580651Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.580666Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 12 partNo 0 count 1 size 262204
2025-11-28T18:22:31.582428Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 12 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.582453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.582482Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.582485Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.582494Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.582501Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 12, partNo: 0, Offset: 12 is stored on disk
2025-11-28T18:22:31.582534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.582540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.582542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.582544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.582547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.582555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.582566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.582655Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.582943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077375:3383] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 12
    Offset: 12
    WriteTimestampMS: 1764354151579
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.583766Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.584093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077379:3385], now have 1 active actors on pipe
2025-11-28T18:22:31.584207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.584215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.584246Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|c07bd521-b954773c-56c838ea-a0fcf157_13 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.584257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.584261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.584265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.584271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.584274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.584283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.584290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.584292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.584303Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.584317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.584370Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.584639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077379:3385] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|c07bd521-b954773c-56c838ea-a0fcf157_13"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|c07bd521-b954773c-56c838ea-a0fcf157_13"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 13
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.586828Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.587308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077382:3386], now have 1 active actors on pipe
2025-11-28T18:22:31.587693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.587706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.587728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 13 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.587738Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.587751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.587755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.587765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.587767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.587772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.588785Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 0
2025-11-28T18:22:31.588819Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 13 partNo 0 FormedBlobsCount 0 NewHead: Offset 13 PartNo 0 PackedSize 262214 count 1 nextOffset 14 batches 1
2025-11-28T18:22:31.588828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.588830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.588833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.588930Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 13,1 HeadOffset 13 endOffset 13 curOffset 14 d0000000000_00000000000000000013_00000_0000000001_00000? size 262204 WTime 1764354151588
2025-11-28T18:22:31.588976Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.588988Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 13 partNo 0 count 1 size 262204
2025-11-28T18:22:31.591140Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 13 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.591162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.591184Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.591187Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.591198Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.591256Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 13, partNo: 0, Offset: 13 is stored on disk
2025-11-28T18:22:31.591294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.591296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.591299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.591301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.591303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.591312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.591324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.591382Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.591664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077382:3386] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 13
    Offset: 13
    WriteTimestampMS: 1764354151587
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.592384Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.592823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077386:3388], now have 1 active actors on pipe
2025-11-28T18:22:31.592850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.592856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.592890Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|84ab2d0-595bf50c-b2f6e5d5-42d09e4f_14 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.592899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.592902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.592906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.592909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.592912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.592921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.592923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.592926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.592937Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.592947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.593079Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.593318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077386:3388] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|84ab2d0-595bf50c-b2f6e5d5-42d09e4f_14"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|84ab2d0-595bf50c-b2f6e5d5-42d09e4f_14"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 14
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.595564Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.596981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077389:3389], now have 1 active actors on pipe
2025-11-28T18:22:31.597396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.597407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.597429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 14 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.597440Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.597454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.597458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.597468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.597470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.597474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.598434Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 0
2025-11-28T18:22:31.598485Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 14 partNo 0 FormedBlobsCount 0 NewHead: Offset 14 PartNo 0 PackedSize 262214 count 1 nextOffset 15 batches 1
2025-11-28T18:22:31.598495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.598497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.598500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.598597Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 14,1 HeadOffset 14 endOffset 14 curOffset 15 d0000000000_00000000000000000014_00000_0000000001_00000? size 262204 WTime 1764354151598
2025-11-28T18:22:31.598642Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.598654Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 14 partNo 0 count 1 size 262204
2025-11-28T18:22:31.600894Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 14 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.600916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.600936Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.600939Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.600946Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.600955Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 14, partNo: 0, Offset: 14 is stored on disk
2025-11-28T18:22:31.600984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.600986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.600988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.600990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.600992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.601000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.601011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.601089Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.601451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077389:3389] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 14
    Offset: 14
    WriteTimestampMS: 1764354151597
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 0
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.602246Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.602719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077393:3391], now have 1 active actors on pipe
2025-11-28T18:22:31.602828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.602837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.602868Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|fc9f0889-57a8278d-a2f7336a-e0c457fb_15 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.602876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.602879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.602883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.602885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.602889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.602898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.602900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.602903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.602915Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.602926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.603043Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.603286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077393:3391] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|fc9f0889-57a8278d-a2f7336a-e0c457fb_15"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|fc9f0889-57a8278d-a2f7336a-e0c457fb_15"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 15
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.605330Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.605949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077396:3392], now have 1 active actors on pipe
2025-11-28T18:22:31.606069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.606075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.606091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 15 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.606099Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.606109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.606112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.606119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.606120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.606123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.606725Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 0
2025-11-28T18:22:31.606759Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 15 partNo 0 FormedBlobsCount 0 NewHead: Offset 15 PartNo 0 PackedSize 262214 count 1 nextOffset 16 batches 1
2025-11-28T18:22:31.606770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.606773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.606775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.606843Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 15,1 HeadOffset 15 endOffset 15 curOffset 16 d0000000000_00000000000000000015_00000_0000000001_00000? size 262204 WTime 1764354151606
2025-11-28T18:22:31.606884Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.606898Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 15 partNo 0 count 1 size 262204
2025-11-28T18:22:31.609085Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 15 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.609106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.609149Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.610531Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:496: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-11-28T18:22:31.610560Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.610570Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.610585Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 15, partNo: 0, Offset: 15 is stored on disk
2025-11-28T18:22:31.610628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.610635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.610638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.610640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.610642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.610652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.610664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.610757Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.611165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077396:3392] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 15
    Offset: 15
    WriteTimestampMS: 1764354151602
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 3
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.611995Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.612479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077400:3394], now have 1 active actors on pipe
2025-11-28T18:22:31.612696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.612711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.612758Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|49460554-f0e221d-4f18e308-5a1cb8a5_16 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.612772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.612776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.612781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.612788Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.612792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.612801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.612807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.612810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.612823Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.612841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.612900Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|49460554-f0e221d-4f18e308-5a1cb8a5_16"
    Status: Active
    SeqNo: 0
  }
}
2025-11-28T18:22:31.613241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077400:3394] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|49460554-f0e221d-4f18e308-5a1cb8a5_16"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 16
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.615196Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.615663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077404:3396], now have 1 active actors on pipe
2025-11-28T18:22:31.616078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.616094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.616114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 16 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.616130Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.616148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.616155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.616165Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.616171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.616175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.617177Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 0
2025-11-28T18:22:31.617226Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 16 partNo 0 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 262214 count 1 nextOffset 17 batches 1
2025-11-28T18:22:31.617240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.617247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.617251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.617352Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 16,1 HeadOffset 16 endOffset 16 curOffset 17 d0000000000_00000000000000000016_00000_0000000001_00000? size 262204 WTime 1764354151617
2025-11-28T18:22:31.617401Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.617420Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 16 partNo 0 count 1 size 262204
2025-11-28T18:22:31.619707Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 16 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.619730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.619751Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.619753Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.619760Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.619768Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 16, partNo: 0, Offset: 16 is stored on disk
2025-11-28T18:22:31.619802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.619808Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.619810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.619812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.619827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.619838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.619853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.619926Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.620276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077404:3396] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 16
    Offset: 16
    WriteTimestampMS: 1764354151615
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.620983Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.621342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077408:3398], now have 1 active actors on pipe
2025-11-28T18:22:31.621489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.621499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.621532Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f5817487-8a114e14-e43090d2-fc581444_17 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.621544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.621548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.621557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.621563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.621567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.621580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.621587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.621590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.621607Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.621623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.621683Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.621915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077408:3398] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f5817487-8a114e14-e43090d2-fc581444_17"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f5817487-8a114e14-e43090d2-fc581444_17"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 17
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.624049Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.625600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077411:3399], now have 1 active actors on pipe
2025-11-28T18:22:31.625988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.626004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.626027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 17 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.626038Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.626052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.626060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.626071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.626077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.626080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.627064Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 0
2025-11-28T18:22:31.627104Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 17 partNo 0 FormedBlobsCount 0 NewHead: Offset 17 PartNo 0 PackedSize 262214 count 1 nextOffset 18 batches 1
2025-11-28T18:22:31.627117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.627120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.627123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.627214Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 17,1 HeadOffset 17 endOffset 17 curOffset 18 d0000000000_00000000000000000017_00000_0000000001_00000? size 262204 WTime 1764354151627
2025-11-28T18:22:31.627261Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.627277Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 17 partNo 0 count 1 size 262204
2025-11-28T18:22:31.629166Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 17 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.629185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.629204Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.629215Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.629228Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 17, partNo: 0, Offset: 17 is stored on disk
2025-11-28T18:22:31.629260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.629268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.629270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.629273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.629275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.629282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.629293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.629325Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.629392Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.629826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077411:3399] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 17
    Offset: 17
    WriteTimestampMS: 1764354151625
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.630678Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.631138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077415:3401], now have 1 active actors on pipe
2025-11-28T18:22:31.631255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.631265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.631298Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|67c8eb8e-22f51142-bdb8fc82-3e8adae7_18 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.631310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.631314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.631323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.631325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.631328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.631338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.631344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.631347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.631359Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.631374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.631439Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|67c8eb8e-22f51142-bdb8fc82-3e8adae7_18"
    Status: Active
    SeqNo: 0
  }
}
2025-11-28T18:22:31.631788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077415:3401] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|67c8eb8e-22f51142-bdb8fc82-3e8adae7_18"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 18
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.634007Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.634377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077418:3402], now have 1 active actors on pipe
2025-11-28T18:22:31.634765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.634781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.634803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 18 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.634817Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.634830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.634838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.634847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.634853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.634858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.635989Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 0
2025-11-28T18:22:31.636032Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 18 partNo 0 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 262214 count 1 nextOffset 19 batches 1
2025-11-28T18:22:31.636045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.636049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.636054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.636148Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 18,1 HeadOffset 18 endOffset 18 curOffset 19 d0000000000_00000000000000000018_00000_0000000001_00000? size 262204 WTime 1764354151635
2025-11-28T18:22:31.636200Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.636216Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 18 partNo 0 count 1 size 262204
2025-11-28T18:22:31.638325Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 18 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.638350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.638369Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.638371Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.638379Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.638393Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 18, partNo: 0, Offset: 18 is stored on disk
2025-11-28T18:22:31.638427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.638435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.638437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.638440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.638442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.638449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.638482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.638548Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.638863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077418:3402] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 18
    Offset: 18
    WriteTimestampMS: 1764354151634
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.639721Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.640115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077425:3404], now have 1 active actors on pipe
2025-11-28T18:22:31.640323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.640336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.640369Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|25b32be3-aecb400c-90618357-6ffc15df_19 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.640382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.640385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.640391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.640392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.640396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.640404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.640406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.640409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.640422Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.640439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.640511Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.640753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077425:3404] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|25b32be3-aecb400c-90618357-6ffc15df_19"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|25b32be3-aecb400c-90618357-6ffc15df_19"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 19
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.643449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.643462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.643464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.643469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.643471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.643762Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.644158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077428:3405], now have 1 active actors on pipe
2025-11-28T18:22:31.644578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.644593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.644615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 19 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.644624Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.644645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.644653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.644663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.644665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.644669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.645666Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 0
2025-11-28T18:22:31.645714Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 19 partNo 0 FormedBlobsCount 0 NewHead: Offset 19 PartNo 0 PackedSize 262214 count 1 nextOffset 20 batches 1
2025-11-28T18:22:31.645728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.645731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.645734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.645831Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 19,1 HeadOffset 19 endOffset 19 curOffset 20 d0000000000_00000000000000000019_00000_0000000001_00000? size 262204 WTime 1764354151645
2025-11-28T18:22:31.645884Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.645902Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 19 partNo 0 count 1 size 262204
2025-11-28T18:22:31.648162Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 19 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.648189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.648208Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.648208Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.648222Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.648231Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 19, partNo: 0, Offset: 19 is stored on disk
2025-11-28T18:22:31.648262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.648269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.648271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.648272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.648274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.648281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.648292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.648364Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.648744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077428:3405] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 19
    Offset: 19
    WriteTimestampMS: 1764354151644
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.649560Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.649991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077432:3407], now have 1 active actors on pipe
2025-11-28T18:22:31.650091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.650103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.650135Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|89163f0c-583efa8b-9906d8c4-9b23f5d2_20 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.650149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.650152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.650156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.650158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.650162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.650171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.650178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.650181Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.650192Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.650208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.650270Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|89163f0c-583efa8b-9906d8c4-9b23f5d2_20"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
2025-11-28T18:22:31.650626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077432:3407] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|89163f0c-583efa8b-9906d8c4-9b23f5d2_20"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 20
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.652547Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.652916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077435:3408], now have 1 active actors on pipe
2025-11-28T18:22:31.653362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.653377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.653399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 20 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.653414Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.653428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.653439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.653449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.653451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.653455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.654388Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 0
2025-11-28T18:22:31.654435Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 20 partNo 0 FormedBlobsCount 0 NewHead: Offset 20 PartNo 0 PackedSize 262214 count 1 nextOffset 21 batches 1
2025-11-28T18:22:31.654449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.654451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.654454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.654660Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 20,1 HeadOffset 20 endOffset 20 curOffset 21 d0000000000_00000000000000000020_00000_0000000001_00000? size 262204 WTime 1764354151654
2025-11-28T18:22:31.654709Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.654724Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 20 partNo 0 count 1 size 262204
2025-11-28T18:22:31.656844Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 20 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.656865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.656884Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.656896Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.656906Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 20, partNo: 0, Offset: 20 is stored on disk
2025-11-28T18:22:31.656941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.656947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.656949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.656952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.656954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.656961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.656972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.657015Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.657090Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.657522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077435:3408] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 20
    Offset: 20
    WriteTimestampMS: 1764354151653
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.658287Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.658748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077439:3410], now have 1 active actors on pipe
2025-11-28T18:22:31.658920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.658931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.658963Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|da11b767-374262de-8c8eeeb5-955e3694_21 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.658975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.658978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.658982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.658984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.658987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.658996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.659004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.659006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.659019Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.659034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.659141Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|da11b767-374262de-8c8eeeb5-955e3694_21"
    Status: Active
    SeqNo: 0
  }
}
2025-11-28T18:22:31.659459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077439:3410] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|da11b767-374262de-8c8eeeb5-955e3694_21"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 21
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.661470Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.662172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077442:3411], now have 1 active actors on pipe
2025-11-28T18:22:31.662262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.662272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.662293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 21 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.662306Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.662323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.662331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.662340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.662343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.662347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.663336Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 0
2025-11-28T18:22:31.663379Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 21 partNo 0 FormedBlobsCount 0 NewHead: Offset 21 PartNo 0 PackedSize 262214 count 1 nextOffset 22 batches 1
2025-11-28T18:22:31.663391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.663395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.663397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.663490Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 21,1 HeadOffset 21 endOffset 21 curOffset 22 d0000000000_00000000000000000021_00000_0000000001_00000? size 262204 WTime 1764354151663
2025-11-28T18:22:31.663541Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.663558Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 21 partNo 0 count 1 size 262204
2025-11-28T18:22:31.665805Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 21 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.665827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.665847Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.665857Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.665866Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 21, partNo: 0, Offset: 21 is stored on disk
2025-11-28T18:22:31.665897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.665904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.665906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.665909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.665911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.665919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.665930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.665968Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.666039Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.666406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077442:3411] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 21
    Offset: 21
    WriteTimestampMS: 1764354151662
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.667092Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.667584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077446:3413], now have 1 active actors on pipe
2025-11-28T18:22:31.667603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.667609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.667642Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|7805ef8a-54c9b5f7-291337b-c3d9cd5b_22 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.667651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.667654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.667659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.667661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.667664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.667672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.667680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.667683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.667695Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.667707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.667798Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|7805ef8a-54c9b5f7-291337b-c3d9cd5b_22"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
2025-11-28T18:22:31.668139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077446:3413] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|7805ef8a-54c9b5f7-291337b-c3d9cd5b_22"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 22
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.670049Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.670806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077449:3414], now have 1 active actors on pipe
2025-11-28T18:22:31.670893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.670904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.670922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 22 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.670930Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.670942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.670949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.670958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.670960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.670964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.672029Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 0
2025-11-28T18:22:31.672067Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 22 partNo 0 FormedBlobsCount 0 NewHead: Offset 22 PartNo 0 PackedSize 262214 count 1 nextOffset 23 batches 1
2025-11-28T18:22:31.672080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.672082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.672085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.672169Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 22,1 HeadOffset 22 endOffset 22 curOffset 23 d0000000000_00000000000000000022_00000_0000000001_00000? size 262204 WTime 1764354151672
2025-11-28T18:22:31.672213Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.672227Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 22 partNo 0 count 1 size 262204
2025-11-28T18:22:31.674634Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.674412Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 22 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.674432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.674451Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.674478Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.674486Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 22, partNo: 0, Offset: 22 is stored on disk
2025-11-28T18:22:31.674517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.674519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.674522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.674527Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.674529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.674534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.674543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.674549Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 0 suffix '63' size 262204
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 22
    Offset: 22
    WriteTimestampMS: 1764354151670
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.675051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077449:3414] destroyed
2025-11-28T18:22:31.675475Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.675907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077453:3416], now have 1 active actors on pipe
2025-11-28T18:22:31.675984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.675993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.676021Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|51400628-8f120683-59876d8e-40dd2dbf_23 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.676032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.676035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.676040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.676046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.676050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.676056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.676059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.676063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.676072Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.676086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.676146Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|51400628-8f120683-59876d8e-40dd2dbf_23"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|51400628-8f120683-59876d8e-40dd2dbf_23"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 23
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.677926Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.678564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077453:3416] destroyed
2025-11-28T18:22:31.678582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077456:3417], now have 1 active actors on pipe
2025-11-28T18:22:31.678679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.678684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.678703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 23 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.678715Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.678727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.678729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.678738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.678740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.678744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.679698Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 0
2025-11-28T18:22:31.679734Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 23 partNo 0 FormedBlobsCount 0 NewHead: Offset 23 PartNo 0 PackedSize 262214 count 1 nextOffset 24 batches 1
2025-11-28T18:22:31.679743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.679745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.679751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.679847Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 23,1 HeadOffset 23 endOffset 23 curOffset 24 d0000000000_00000000000000000023_00000_0000000001_00000? size 262204 WTime 1764354151679
2025-11-28T18:22:31.679892Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.679905Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 23 partNo 0 count 1 size 262204
2025-11-28T18:22:31.682017Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 23 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.682042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.682059Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.682063Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.682068Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.682079Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 23, partNo: 0, Offset: 23 is stored on disk
2025-11-28T18:22:31.682105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.682112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.682114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.682116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.682118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.682124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.682133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.682206Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 23
    Offset: 23
    WriteTimestampMS: 1764354151678
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:31.682547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077456:3417] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.683046Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.683490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077460:3419], now have 1 active actors on pipe
2025-11-28T18:22:31.683567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.683575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.683607Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|7c2b2803-3981cc20-71d698cb-7f3e04c3_24 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.683614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.683617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.683755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.683759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.683763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.683770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.683773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.683775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.683785Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.683796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
2025-11-28T18:22:31.683875Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|7c2b2803-3981cc20-71d698cb-7f3e04c3_24"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
2025-11-28T18:22:31.684113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077460:3419] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|7c2b2803-3981cc20-71d698cb-7f3e04c3_24"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 24
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.685944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077463:3420], now have 1 active actors on pipe
2025-11-28T18:22:31.685573Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.686369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.686380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.686398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 24 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.686410Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.686419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.686421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.686429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.686431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.686435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.687421Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 0
2025-11-28T18:22:31.687467Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 24 partNo 0 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 262214 count 1 nextOffset 25 batches 1
2025-11-28T18:22:31.687476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.687482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.687485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.687575Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 24,1 HeadOffset 24 endOffset 24 curOffset 25 d0000000000_00000000000000000024_00000_0000000001_00000? size 262204 WTime 1764354151687
2025-11-28T18:22:31.687622Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.687638Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 24 partNo 0 count 1 size 262204
2025-11-28T18:22:31.689553Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 24 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.689581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.689593Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.689600Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.689608Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.689617Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 24, partNo: 0, Offset: 24 is stored on disk
2025-11-28T18:22:31.689648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.689657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.689659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.689661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.689664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.689671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.689680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.689751Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 24
    Offset: 24
    WriteTimestampMS: 1764354151686
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.690070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077463:3420] destroyed
2025-11-28T18:22:31.690562Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.690847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077469:3423], now have 1 active actors on pipe
2025-11-28T18:22:31.690966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.690975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.691008Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|32f67f14-3bbc8211-6a9da183-3014fb49_25 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.691015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.691018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.691023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.691024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.691027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.691036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.691040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.691043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.691053Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.691063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.691207Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.691445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077469:3423] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|32f67f14-3bbc8211-6a9da183-3014fb49_25"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|32f67f14-3bbc8211-6a9da183-3014fb49_25"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 25
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.693409Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.693715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077474:3425], now have 1 active actors on pipe
2025-11-28T18:22:31.694190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.694208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.694234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 25 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.694250Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.694264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.694273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.694285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.694287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.694291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.695241Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 0
2025-11-28T18:22:31.695294Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 25 partNo 0 FormedBlobsCount 0 NewHead: Offset 25 PartNo 0 PackedSize 262214 count 1 nextOffset 26 batches 1
2025-11-28T18:22:31.695311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.695315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.695319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.695422Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 25,1 HeadOffset 25 endOffset 25 curOffset 26 d0000000000_00000000000000000025_00000_0000000001_00000? size 262204 WTime 1764354151695
2025-11-28T18:22:31.695479Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.695497Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 25 partNo 0 count 1 size 262204
2025-11-28T18:22:31.703130Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 25 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.703164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.703195Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.703206Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.703225Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 25, partNo: 0, Offset: 25 is stored on disk
2025-11-28T18:22:31.703272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.703281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.703284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.703286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.703288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.703300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.703316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.703360Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.703439Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.703903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077474:3425] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 25
    Offset: 25
    WriteTimestampMS: 1764354151694
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 7
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.704624Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.705289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077478:3427], now have 1 active actors on pipe
2025-11-28T18:22:31.705321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.705329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.705360Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|79f2f5fa-70a233d9-2184de08-4dab5a65_26 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.705376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.705380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.705385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.705388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.705391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.705404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.705407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.705410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.705425Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.705446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.705534Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.705873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077478:3427] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|79f2f5fa-70a233d9-2184de08-4dab5a65_26"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|79f2f5fa-70a233d9-2184de08-4dab5a65_26"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 26
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.708006Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.708547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077481:3428], now have 1 active actors on pipe
2025-11-28T18:22:31.708996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.709006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.709030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 26 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.709044Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.709056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.709060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.709072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.709074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.709079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.710504Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 0
2025-11-28T18:22:31.710560Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 26 partNo 0 FormedBlobsCount 0 NewHead: Offset 26 PartNo 0 PackedSize 262214 count 1 nextOffset 27 batches 1
2025-11-28T18:22:31.710572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.710575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.710578Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.710698Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 26,1 HeadOffset 26 endOffset 26 curOffset 27 d0000000000_00000000000000000026_00000_0000000001_00000? size 262204 WTime 1764354151710
2025-11-28T18:22:31.710749Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.710764Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 26 partNo 0 count 1 size 262204
2025-11-28T18:22:31.713136Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 26 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.713158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.713182Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.713194Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.713204Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 26, partNo: 0, Offset: 26 is stored on disk
2025-11-28T18:22:31.713245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.713247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.713249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.713252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.713255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.713266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.713278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.713398Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.713503Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.713889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077481:3428] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 26
    Offset: 26
    WriteTimestampMS: 1764354151708
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.714542Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.715143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077485:3430], now have 1 active actors on pipe
2025-11-28T18:22:31.715287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.715296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.715326Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|78afa1a-3fed3906-a1b3845a-8b20203a_27 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.715336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.715339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.715345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.715347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.715350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.715365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.715368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.715370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.715386Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.715400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.715544Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|78afa1a-3fed3906-a1b3845a-8b20203a_27"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
2025-11-28T18:22:31.715795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077485:3430] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|78afa1a-3fed3906-a1b3845a-8b20203a_27"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 27
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.717494Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.719570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077488:3431], now have 1 active actors on pipe
2025-11-28T18:22:31.720020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.720033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.720055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 27 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.720066Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.720079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.720083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.720093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.720095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.720100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.721247Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 0
2025-11-28T18:22:31.721291Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 27 partNo 0 FormedBlobsCount 0 NewHead: Offset 27 PartNo 0 PackedSize 262214 count 1 nextOffset 28 batches 1
2025-11-28T18:22:31.721303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.721306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.721310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.721419Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 27,1 HeadOffset 27 endOffset 27 curOffset 28 d0000000000_00000000000000000027_00000_0000000001_00000? size 262204 WTime 1764354151721
2025-11-28T18:22:31.721469Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.721482Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 27 partNo 0 count 1 size 262204
2025-11-28T18:22:31.724077Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 27 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.724092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.724110Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.724118Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.724125Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 27, partNo: 0, Offset: 27 is stored on disk
2025-11-28T18:22:31.724154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.724156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.724157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.724158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.724159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.724166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.724176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.724264Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.724338Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 27
    Offset: 27
    WriteTimestampMS: 1764354151719
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:31.724767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077488:3431] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.725364Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.725896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077492:3433], now have 1 active actors on pipe
2025-11-28T18:22:31.725909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.725915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.725941Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|33eff583-11aa9a87-6041d3aa-fa296516_28 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.725949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.725952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.725957Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.725959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.725963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.725971Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.725973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.725975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.725984Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.725994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.726104Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|33eff583-11aa9a87-6041d3aa-fa296516_28"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
2025-11-28T18:22:31.726411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077492:3433] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|33eff583-11aa9a87-6041d3aa-fa296516_28"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 28
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.728274Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.728861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077495:3434], now have 1 active actors on pipe
2025-11-28T18:22:31.729027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.729046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.729069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 28 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.729084Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.729105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.729114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.729124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.729127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.729132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.730235Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 0
2025-11-28T18:22:31.730285Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 28 partNo 0 FormedBlobsCount 0 NewHead: Offset 28 PartNo 0 PackedSize 262214 count 1 nextOffset 29 batches 1
2025-11-28T18:22:31.730302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.730305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.730309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.730405Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 28,1 HeadOffset 28 endOffset 28 curOffset 29 d0000000000_00000000000000000028_00000_0000000001_00000? size 262204 WTime 1764354151729
2025-11-28T18:22:31.730479Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.730499Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 28 partNo 0 count 1 size 262204
2025-11-28T18:22:31.732850Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 28 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.732915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.732938Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.732951Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.732964Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 28, partNo: 0, Offset: 28 is stored on disk
2025-11-28T18:22:31.732972Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.732994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.732997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.732999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.733001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.733003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.733010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.733022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 28
    Offset: 28
    WriteTimestampMS: 1764354151728
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.733115Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.733448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077495:3434] destroyed
2025-11-28T18:22:31.734207Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.734923Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|68f3685e-ccd36b20-12c2896a-d2778dfc_29"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|68f3685e-ccd36b20-12c2896a-d2778dfc_29"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 29
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.734608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077501:3438], now have 1 active actors on pipe
2025-11-28T18:22:31.734759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.734764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.734793Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|68f3685e-ccd36b20-12c2896a-d2778dfc_29 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.734802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.734805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.734810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.734811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.734815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.734823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.734825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.734827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.734841Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.734852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.736831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077501:3438] destroyed
2025-11-28T18:22:31.736900Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.737353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077504:3439], now have 1 active actors on pipe
2025-11-28T18:22:31.737845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.737857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.737879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 29 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.737889Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.737903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.737907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.737918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.737920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.737925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.738803Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 0
2025-11-28T18:22:31.738849Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 29 partNo 0 FormedBlobsCount 0 NewHead: Offset 29 PartNo 0 PackedSize 262214 count 1 nextOffset 30 batches 1
2025-11-28T18:22:31.738860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.738862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.738865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.738958Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 29,1 HeadOffset 29 endOffset 29 curOffset 30 d0000000000_00000000000000000029_00000_0000000001_00000? size 262204 WTime 1764354151738
2025-11-28T18:22:31.739017Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.739029Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 29 partNo 0 count 1 size 262204
2025-11-28T18:22:31.741491Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 29 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.741517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.741537Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.741548Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.741566Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 29, partNo: 0, Offset: 29 is stored on disk
2025-11-28T18:22:31.741606Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.741616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.741618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.741621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.741623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.741633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.741649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.741706Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.741807Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 29
    Offset: 29
    WriteTimestampMS: 1764354151737
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.743039Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.743281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077504:3439] destroyed
2025-11-28T18:22:31.743668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077508:3441], now have 1 active actors on pipe
2025-11-28T18:22:31.743680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.743686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.743714Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|437cd15c-4b232b4f-9aea657f-a2d57a39_30 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.743724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.743728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.743732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.743734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.743739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.743748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.743751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.743754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.743768Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.743776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.743778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.743780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.743783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.743787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.743797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.743868Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|437cd15c-4b232b4f-9aea657f-a2d57a39_30"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|437cd15c-4b232b4f-9aea657f-a2d57a39_30"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 30
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.744186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077508:3441] destroyed
2025-11-28T18:22:31.746073Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.746528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077511:3442], now have 1 active actors on pipe
2025-11-28T18:22:31.746911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.746925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.746948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 30 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.746959Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.746972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.746976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.746987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.746989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.746994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.747870Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 0
2025-11-28T18:22:31.747908Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 30 partNo 0 FormedBlobsCount 0 NewHead: Offset 30 PartNo 0 PackedSize 262214 count 1 nextOffset 31 batches 1
2025-11-28T18:22:31.747918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.747921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.747927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.748020Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 30,1 HeadOffset 30 endOffset 30 curOffset 31 d0000000000_00000000000000000030_00000_0000000001_00000? size 262204 WTime 1764354151747
2025-11-28T18:22:31.748069Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.748081Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 30 partNo 0 count 1 size 262204
2025-11-28T18:22:31.750444Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 30 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.750483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.750506Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.750516Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.750536Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.750538Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 30, partNo: 0, Offset: 30 is stored on disk
2025-11-28T18:22:31.750576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.750579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.750581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.750583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.750585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.750596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.750609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.750699Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 30
    Offset: 30
    WriteTimestampMS: 1764354151746
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.751872Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.752232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077511:3442] destroyed
2025-11-28T18:22:31.752444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077518:3444], now have 1 active actors on pipe
2025-11-28T18:22:31.752588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.752597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.752631Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|8b729dff-f659864c-9903cf62-c5ec2ca_31 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.752643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.752662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.752667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.752669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.752672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.752681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.752683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.752686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.752698Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.752710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|8b729dff-f659864c-9903cf62-c5ec2ca_31"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
2025-11-28T18:22:31.752765Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.753173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077518:3444] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|8b729dff-f659864c-9903cf62-c5ec2ca_31"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 31
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.755289Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.755788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077521:3445], now have 1 active actors on pipe
2025-11-28T18:22:31.756212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.756229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.756254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 31 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.756279Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.756299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.756307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.756318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.756320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.756324Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.757359Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 0
2025-11-28T18:22:31.757458Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 31 partNo 0 FormedBlobsCount 0 NewHead: Offset 31 PartNo 0 PackedSize 262214 count 1 nextOffset 32 batches 1
2025-11-28T18:22:31.757467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.757470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.757473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.757581Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 31,1 HeadOffset 31 endOffset 31 curOffset 32 d0000000000_00000000000000000031_00000_0000000001_00000? size 262204 WTime 1764354151757
2025-11-28T18:22:31.757629Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.757640Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 31 partNo 0 count 1 size 262204
2025-11-28T18:22:31.760412Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 31
    Offset: 31
    WriteTimestampMS: 1764354151756
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.761581Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.760095Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 31 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.760124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.760146Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.760150Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.760155Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.760166Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 31, partNo: 0, Offset: 31 is stored on disk
2025-11-28T18:22:31.760200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.760203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.760205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.760207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.760209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.760220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000000_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000001_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000002_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000003_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000004_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000005_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000006_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000007_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000008_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000009_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000010_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000011_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000012_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000013_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000014_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000015_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000016_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000017_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000018_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000019_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000023_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000024_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000025_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000026_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000027_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000028_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000029_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000030_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000031_00000_0000000001_00000? (262204)
2025-11-28T18:22:31.760312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 32 keys were taken away. Let's read 8390528 bytes
2025-11-28T18:22:31.760327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.760348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 32 blobs
2025-11-28T18:22:31.760351Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000000_00000_0000000001_00000?
2025-11-28T18:22:31.760360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000001_00000_0000000001_00000?
2025-11-28T18:22:31.760371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000002_00000_0000000001_00000?
2025-11-28T18:22:31.760385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000003_00000_0000000001_00000?
2025-11-28T18:22:31.760392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000004_00000_0000000001_00000?
2025-11-28T18:22:31.760415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000005_00000_0000000001_00000?
2025-11-28T18:22:31.760423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000006_00000_0000000001_00000?
2025-11-28T18:22:31.760430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000007_00000_0000000001_00000?
2025-11-28T18:22:31.760440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000008_00000_0000000001_00000?
2025-11-28T18:22:31.760485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000009_00000_0000000001_00000?
2025-11-28T18:22:31.760493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000010_00000_0000000001_00000?
2025-11-28T18:22:31.760502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000011_00000_0000000001_00000?
2025-11-28T18:22:31.760510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000012_00000_0000000001_00000?
2025-11-28T18:22:31.760533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000013_00000_0000000001_00000?
2025-11-28T18:22:31.760542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000014_00000_0000000001_00000?
2025-11-28T18:22:31.760552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000015_00000_0000000001_00000?
2025-11-28T18:22:31.760559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000016_00000_0000000001_00000?
2025-11-28T18:22:31.760626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000017_00000_0000000001_00000?
2025-11-28T18:22:31.760634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000018_00000_0000000001_00000?
2025-11-28T18:22:31.760642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000019_00000_0000000001_00000?
2025-11-28T18:22:31.760650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00000_0000000001_00000?
2025-11-28T18:22:31.760658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00000?
2025-11-28T18:22:31.760666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000001_00000?
2025-11-28T18:22:31.760674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000023_00000_0000000001_00000?
2025-11-28T18:22:31.760683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000024_00000_0000000001_00000?
2025-11-28T18:22:31.760691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000025_00000_0000000001_00000?
2025-11-28T18:22:31.760698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000026_00000_0000000001_00000?
2025-11-28T18:22:31.760707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000027_00000_0000000001_00000?
2025-11-28T18:22:31.760716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000028_00000_0000000001_00000?
2025-11-28T18:22:31.760742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000029_00000_0000000001_00000?
2025-11-28T18:22:31.760751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000030_00000_0000000001_00000?
2025-11-28T18:22:31.760758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000031_00000_0000000001_00000?
2025-11-28T18:22:31.760767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 32 blobs for compaction
2025-11-28T18:22:31.760778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077521:3445] destroyed
2025-11-28T18:22:31.760790Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 1 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760793Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 1 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760797Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 2 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760800Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 3 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760803Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 4 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760806Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 5 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760809Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 6 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760812Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 7 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760816Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 8 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760819Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 9 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760822Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 10 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760826Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 11 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760829Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 12 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760840Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 13 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760846Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 14 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760849Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 15 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760853Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 16 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760856Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 17 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760859Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 18 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760862Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 19 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760866Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 20 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760869Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 21 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760872Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 22 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760875Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 23 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760878Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 24 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760882Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 25 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 26 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760890Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 27 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760893Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 28 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760896Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 29 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760899Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 30 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760902Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 31 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:31.760923Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 32 blobs are from cache.
2025-11-28T18:22:31.760938Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.760943Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.760947Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.760951Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.760955Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.760959Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.760964Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.760968Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.760972Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.760976Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.760978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-28T18:22:31.760981Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.760985Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.760987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/32] d0000000000_00000000000000000000_00000_0000000001_00000?
2025-11-28T18:22:31.760988Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.760989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.760992Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.760993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000000_00000_0000000001_00000?, parameters.CurOffset=0
2025-11-28T18:22:31.760996Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.761000Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.761004Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.761010Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.761013Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.761016Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.761020Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.761023Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.761027Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.761030Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.761033Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.761037Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.761040Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.761043Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.761046Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.761050Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.761054Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.761058Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:31.761086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 0.0/1
2025-11-28T18:22:31.761094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 0
2025-11-28T18:22:31.761123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 0 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 262214 count 1 nextOffset 1 batches 1
2025-11-28T18:22:31.761128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 0:0 appended
2025-11-28T18:22:31.761132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/32] d0000000000_00000000000000000001_00000_0000000001_00000?
2025-11-28T18:22:31.761134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.761136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000001_00000_0000000001_00000?, parameters.CurOffset=1
2025-11-28T18:22:31.761203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 1.0/1
2025-11-28T18:22:31.761206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 0
2025-11-28T18:22:31.761733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 1 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 524387 count 2 nextOffset 2 batches 1
2025-11-28T18:22:31.761738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 1:0 appended
2025-11-28T18:22:31.761743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/32] d0000000000_00000000000000000002_00000_0000000001_00000?
2025-11-28T18:22:31.761745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.761747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000002_00000_0000000001_00000?, parameters.CurOffset=2
2025-11-28T18:22:31.761830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 2.0/1
2025-11-28T18:22:31.761834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 0
2025-11-28T18:22:31.761846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 2 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 786601 count 3 nextOffset 3 batches 2
2025-11-28T18:22:31.761848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 2:0 appended
2025-11-28T18:22:31.761852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/32] d0000000000_00000000000000000003_00000_0000000001_00000?
2025-11-28T18:22:31.761854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.761857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000003_00000_0000000001_00000?, parameters.CurOffset=3
2025-11-28T18:22:31.762103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 3.0/1
2025-11-28T18:22:31.762107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 0
2025-11-28T18:22:31.762534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 3 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1048774 count 4 nextOffset 4 batches 2
2025-11-28T18:22:31.762539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 3:0 appended
2025-11-28T18:22:31.762545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/32] d0000000000_00000000000000000004_00000_0000000001_00000?
2025-11-28T18:22:31.762549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.762557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000004_00000_0000000001_00000?, parameters.CurOffset=4
2025-11-28T18:22:31.762622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 4.0/1
2025-11-28T18:22:31.762626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 0
2025-11-28T18:22:31.762647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 4 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1310988 count 5 nextOffset 5 batches 3
2025-11-28T18:22:31.762649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 4:0 appended
2025-11-28T18:22:31.762653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/32] d0000000000_00000000000000000005_00000_0000000001_00000?
2025-11-28T18:22:31.762655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.762657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000005_00000_0000000001_00000?, parameters.CurOffset=5
2025-11-28T18:22:31.762736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 5.0/1
2025-11-28T18:22:31.762740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 0
2025-11-28T18:22:31.763350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 5 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1573161 count 6 nextOffset 6 batches 3
2025-11-28T18:22:31.763355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 5:0 appended
2025-11-28T18:22:31.763362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/32] d0000000000_00000000000000000006_00000_0000000001_00000?
2025-11-28T18:22:31.763364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.763367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000006_00000_0000000001_00000?, parameters.CurOffset=6
2025-11-28T18:22:31.763441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 6.0/1
2025-11-28T18:22:31.763446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 0
2025-11-28T18:22:31.763458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 6 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1835375 count 7 nextOffset 7 batches 4
2025-11-28T18:22:31.763460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 6:0 appended
2025-11-28T18:22:31.763463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/32] d0000000000_00000000000000000007_00000_0000000001_00000?
2025-11-28T18:22:31.763465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.763469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000007_00000_0000000001_00000?, parameters.CurOffset=7
2025-11-28T18:22:31.763534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 7.0/1
2025-11-28T18:22:31.763537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 0
2025-11-28T18:22:31.764095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 7 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2097548 count 8 nextOffset 8 batches 4
2025-11-28T18:22:31.764100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 7:0 appended
2025-11-28T18:22:31.764106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[8/32] d0000000000_00000000000000000008_00000_0000000001_00000?
2025-11-28T18:22:31.764108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.764111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000008_00000_0000000001_00000?, parameters.CurOffset=8
2025-11-28T18:22:31.764175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 8.0/1
2025-11-28T18:22:31.764179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 0
2025-11-28T18:22:31.764189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 8 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2359762 count 9 nextOffset 9 batches 5
2025-11-28T18:22:31.764191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 8:0 appended
2025-11-28T18:22:31.764194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[9/32] d0000000000_00000000000000000009_00000_0000000001_00000?
2025-11-28T18:22:31.764196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.764198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000009_00000_0000000001_00000?, parameters.CurOffset=9
2025-11-28T18:22:31.764265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 9.0/1
2025-11-28T18:22:31.764268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 0
2025-11-28T18:22:31.764594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 9 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2621935 count 10 nextOffset 10 batches 5
2025-11-28T18:22:31.764597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 9:0 appended
2025-11-28T18:22:31.764602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[10/32] d0000000000_00000000000000000010_00000_0000000001_00000?
2025-11-28T18:22:31.764604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.764606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000010_00000_0000000001_00000?, parameters.CurOffset=10
2025-11-28T18:22:31.764678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 10.0/1
2025-11-28T18:22:31.764683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 0
2025-11-28T18:22:31.764707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 10 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2884149 count 11 nextOffset 11 batches 6
2025-11-28T18:22:31.764709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 10:0 appended
2025-11-28T18:22:31.764713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[11/32] d0000000000_00000000000000000011_00000_0000000001_00000?
2025-11-28T18:22:31.764715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.764717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000011_00000_0000000001_00000?, parameters.CurOffset=11
2025-11-28T18:22:31.764799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 11.0/1
2025-11-28T18:22:31.764802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 0
2025-11-28T18:22:31.765106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 11 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3146322 count 12 nextOffset 12 batches 6
2025-11-28T18:22:31.765111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 11:0 appended
2025-11-28T18:22:31.765116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[12/32] d0000000000_00000000000000000012_00000_0000000001_00000?
2025-11-28T18:22:31.765118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.765121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000012_00000_0000000001_00000?, parameters.CurOffset=12
2025-11-28T18:22:31.765192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 12.0/1
2025-11-28T18:22:31.765196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 0
2025-11-28T18:22:31.765207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 12 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3408536 count 13 nextOffset 13 batches 7
2025-11-28T18:22:31.765209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 12:0 appended
2025-11-28T18:22:31.765213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[13/32] d0000000000_00000000000000000013_00000_0000000001_00000?
2025-11-28T18:22:31.765215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.765217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000013_00000_0000000001_00000?, parameters.CurOffset=13
2025-11-28T18:22:31.765282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 13.0/1
2025-11-28T18:22:31.765286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 0
2025-11-28T18:22:31.765780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 13 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3670709 count 14 nextOffset 14 batches 7
2025-11-28T18:22:31.765785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 13:0 appended
2025-11-28T18:22:31.765790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[14/32] d0000000000_00000000000000000014_00000_0000000001_00000?
2025-11-28T18:22:31.765792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.765795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000014_00000_0000000001_00000?, parameters.CurOffset=14
2025-11-28T18:22:31.765868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 14.0/1
2025-11-28T18:22:31.765872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 0
2025-11-28T18:22:31.765896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 14 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3932923 count 15 nextOffset 15 batches 8
2025-11-28T18:22:31.765898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 14:0 appended
2025-11-28T18:22:31.765902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[15/32] d0000000000_00000000000000000015_00000_0000000001_00000?
2025-11-28T18:22:31.765905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.765908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000015_00000_0000000001_00000?, parameters.CurOffset=15
2025-11-28T18:22:31.765970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 15.0/1
2025-11-28T18:22:31.765973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 0
2025-11-28T18:22:31.767218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 15 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4195096 count 16 nextOffset 16 batches 8
2025-11-28T18:22:31.767233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 15:0 appended
2025-11-28T18:22:31.767250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[16/32] d0000000000_00000000000000000016_00000_0000000001_00000?
2025-11-28T18:22:31.767253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.767257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000016_00000_0000000001_00000?, parameters.CurOffset=16
2025-11-28T18:22:31.767366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 16.0/1
2025-11-28T18:22:31.767373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 0
2025-11-28T18:22:31.767401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 16 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4457310 count 17 nextOffset 17 batches 9
2025-11-28T18:22:31.767420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 16:0 appended
2025-11-28T18:22:31.767425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[17/32] d0000000000_00000000000000000017_00000_0000000001_00000?
2025-11-28T18:22:31.767427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.767429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000017_00000_0000000001_00000?, parameters.CurOffset=17
2025-11-28T18:22:31.767499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 17.0/1
2025-11-28T18:22:31.767503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 0
2025-11-28T18:22:31.768039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 17 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4719483 count 18 nextOffset 18 batches 9
2025-11-28T18:22:31.768059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 17:0 appended
2025-11-28T18:22:31.768064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[18/32] d0000000000_00000000000000000018_00000_0000000001_00000?
2025-11-28T18:22:31.768067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.768070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000018_00000_0000000001_00000?, parameters.CurOffset=18
2025-11-28T18:22:31.768146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 18.0/1
2025-11-28T18:22:31.768161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 0
2025-11-28T18:22:31.768174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 18 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4981697 count 19 nextOffset 19 batches 10
2025-11-28T18:22:31.768177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 18:0 appended
2025-11-28T18:22:31.768182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[19/32] d0000000000_00000000000000000019_00000_0000000001_00000?
2025-11-28T18:22:31.768184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.768187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000019_00000_0000000001_00000?, parameters.CurOffset=19
2025-11-28T18:22:31.768241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 19.0/1
2025-11-28T18:22:31.768253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 0
2025-11-28T18:22:31.768792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 19 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 5243870 count 20 nextOffset 20 batches 10
2025-11-28T18:22:31.768806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 19:0 appended
2025-11-28T18:22:31.768812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[20/32] d0000000000_00000000000000000020_00000_0000000001_00000?
2025-11-28T18:22:31.768816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.768818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000020_00000_0000000001_00000?, parameters.CurOffset=20
2025-11-28T18:22:31.768882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 20.0/1
2025-11-28T18:22:31.768896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 0
2025-11-28T18:22:31.768908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 20 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 5506084 count 21 nextOffset 21 batches 11
2025-11-28T18:22:31.768911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 20:0 appended
2025-11-28T18:22:31.768915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[21/32] d0000000000_00000000000000000021_00000_0000000001_00000?
2025-11-28T18:22:31.768917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.768921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000021_00000_0000000001_00000?, parameters.CurOffset=21
2025-11-28T18:22:31.768971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 21.0/1
2025-11-28T18:22:31.768984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 0
2025-11-28T18:22:31.769481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 21 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 5768257 count 22 nextOffset 22 batches 11
2025-11-28T18:22:31.769492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 21:0 appended
2025-11-28T18:22:31.769497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[22/32] d0000000000_00000000000000000022_00000_0000000001_00000?
2025-11-28T18:22:31.769499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.769502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000022_00000_0000000001_00000?, parameters.CurOffset=22
2025-11-28T18:22:31.769563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 22.0/1
2025-11-28T18:22:31.769567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 0
2025-11-28T18:22:31.769580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 22 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 6030471 count 23 nextOffset 23 batches 12
2025-11-28T18:22:31.769582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 22:0 appended
2025-11-28T18:22:31.769586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[23/32] d0000000000_00000000000000000023_00000_0000000001_00000?
2025-11-28T18:22:31.769588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.769592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000023_00000_0000000001_00000?, parameters.CurOffset=23
2025-11-28T18:22:31.769644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 23.0/1
2025-11-28T18:22:31.769647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 0
2025-11-28T18:22:31.770141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 23 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 6292644 count 24 nextOffset 24 batches 12
2025-11-28T18:22:31.770157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 23:0 appended
2025-11-28T18:22:31.770163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[24/32] d0000000000_00000000000000000024_00000_0000000001_00000?
2025-11-28T18:22:31.770165Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.770170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000024_00000_0000000001_00000?, parameters.CurOffset=24
2025-11-28T18:22:31.770228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 24.0/1
2025-11-28T18:22:31.770232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 0
2025-11-28T18:22:31.770249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 24 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 6554858 count 25 nextOffset 25 batches 13
2025-11-28T18:22:31.770251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 24:0 appended
2025-11-28T18:22:31.770255Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[25/32] d0000000000_00000000000000000025_00000_0000000001_00000?
2025-11-28T18:22:31.770257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.770259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000025_00000_0000000001_00000?, parameters.CurOffset=25
2025-11-28T18:22:31.770310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 25.0/1
2025-11-28T18:22:31.770316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 0
2025-11-28T18:22:31.770824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 25 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 6817031 count 26 nextOffset 26 batches 13
2025-11-28T18:22:31.770841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 25:0 appended
2025-11-28T18:22:31.770846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[26/32] d0000000000_00000000000000000026_00000_0000000001_00000?
2025-11-28T18:22:31.770848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.770851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000026_00000_0000000001_00000?, parameters.CurOffset=26
2025-11-28T18:22:31.770909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 26.0/1
2025-11-28T18:22:31.770923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 0
2025-11-28T18:22:31.770948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 26 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 7079245 count 27 nextOffset 27 batches 14
2025-11-28T18:22:31.770951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 26:0 appended
2025-11-28T18:22:31.770955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[27/32] d0000000000_00000000000000000027_00000_0000000001_00000?
2025-11-28T18:22:31.770957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.770960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000027_00000_0000000001_00000?, parameters.CurOffset=27
2025-11-28T18:22:31.771013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 27.0/1
2025-11-28T18:22:31.771028Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 0
2025-11-28T18:22:31.771520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 27 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 7341418 count 28 nextOffset 28 batches 14
2025-11-28T18:22:31.771536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 27:0 appended
2025-11-28T18:22:31.771540Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[28/32] d0000000000_00000000000000000028_00000_0000000001_00000?
2025-11-28T18:22:31.771542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.771546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000028_00000_0000000001_00000?, parameters.CurOffset=28
2025-11-28T18:22:31.771613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 28.0/1
2025-11-28T18:22:31.771626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 0
2025-11-28T18:22:31.771640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 28 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 7603632 count 29 nextOffset 29 batches 15
2025-11-28T18:22:31.771643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 28:0 appended
2025-11-28T18:22:31.771648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[29/32] d0000000000_00000000000000000029_00000_0000000001_00000?
2025-11-28T18:22:31.771656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.771658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000029_00000_0000000001_00000?, parameters.CurOffset=29
2025-11-28T18:22:31.771728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 29.0/1
2025-11-28T18:22:31.771739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 0
2025-11-28T18:22:31.772221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 29 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 7865805 count 30 nextOffset 30 batches 15
2025-11-28T18:22:31.772236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 29:0 appended
2025-11-28T18:22:31.772241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[30/32] d0000000000_00000000000000000030_00000_0000000001_00000?
2025-11-28T18:22:31.772243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.772246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000030_00000_0000000001_00000?, parameters.CurOffset=30
2025-11-28T18:22:31.772320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 30.0/1
2025-11-28T18:22:31.772333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 0
2025-11-28T18:22:31.772347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 30 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 8128019 count 31 nextOffset 31 batches 16
2025-11-28T18:22:31.772355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 30:0 appended
2025-11-28T18:22:31.772359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[31/32] d0000000000_00000000000000000031_00000_0000000001_00000?
2025-11-28T18:22:31.772360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:31.772363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000031_00000_0000000001_00000?, parameters.CurOffset=31
2025-11-28T18:22:31.772428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 31.0/1
2025-11-28T18:22:31.772438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 0
2025-11-28T18:22:31.779501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 31 partNo 0 result is x0000000000_00000000000000000000_00000_0000000031_00000 size 8128009
2025-11-28T18:22:31.779533Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1110: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000000_00000_0000000031_00000 new key d0000000000_00000000000000000000_00000_0000000031_00000 size 8128009 WTime 1764354151779
2025-11-28T18:22:31.779573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 31 partNo 0 FormedBlobsCount 1 NewHead: Offset 31 PartNo 0 PackedSize 262214 count 1 nextOffset 32 batches 1
2025-11-28T18:22:31.779579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 31:0 appended
2025-11-28T18:22:31.779696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 31,1 HeadOffset 0 endOffset 0 curOffset 32 d0000000000_00000000000000000031_00000_0000000001_00000| size 262204 WTime 1764354151779
2025-11-28T18:22:31.779781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077525:3447], now have 1 active actors on pipe
2025-11-28T18:22:31.779811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.779823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.779839Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.779856Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 31 size 8128009
2025-11-28T18:22:31.779868Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 31 partNo 0 count 1 size 262204
2025-11-28T18:22:31.779903Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|dd8c4ef-58781b1f-e2431d42-ecaf0e72_32 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.779913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.779916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.779922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.779924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.779928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.779938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.779948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.779951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.779963Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.780009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|dd8c4ef-58781b1f-e2431d42-ecaf0e72_32"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|dd8c4ef-58781b1f-e2431d42-ecaf0e72_32"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 32
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.782011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077525:3447] destroyed
2025-11-28T18:22:31.780077Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.783572Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.784089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077531:3450], now have 1 active actors on pipe
2025-11-28T18:22:31.784474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.784483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.784506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 32 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.784520Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.784535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.784539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.784551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.784553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.784558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.785621Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 32 partNo 0
2025-11-28T18:22:31.785658Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 32 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 262214 count 1 nextOffset 33 batches 1
2025-11-28T18:22:31.785667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.785670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.785674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.785765Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 32,1 HeadOffset 32 endOffset 32 curOffset 33 d0000000000_00000000000000000032_00000_0000000001_00000? size 262204 WTime 1764354151784
2025-11-28T18:22:31.785813Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.785825Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 32 partNo 0 count 1 size 262204
2025-11-28T18:22:31.795830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 0 count 31 size 8128009 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.795855Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 31 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.795875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.795877Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 31 parts 0 suffix '0' size 8128009
2025-11-28T18:22:31.795882Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 0 suffix '124' size 262204
2025-11-28T18:22:31.796015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-28T18:22:31.796028Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.796399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 32 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.796418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.796442Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.796459Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.796463Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 32 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.796469Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 32, partNo: 0, Offset: 32 is stored on disk
2025-11-28T18:22:31.796494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.796497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.796499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.796502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.796504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.796575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.796594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.796610Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.796624Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000000_00000_0000000001_00000?(+) to d0000000000_00000000000000000000_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796626Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000001_00000_0000000001_00000?(+) to d0000000000_00000000000000000001_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796629Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000002_00000_0000000001_00000?(+) to d0000000000_00000000000000000002_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796631Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000003_00000_0000000001_00000?(+) to d0000000000_00000000000000000003_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796633Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000004_00000_0000000001_00000?(+) to d0000000000_00000000000000000004_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796636Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000005_00000_0000000001_00000?(+) to d0000000000_00000000000000000005_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796638Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000006_00000_0000000001_00000?(+) to d0000000000_00000000000000000006_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796641Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000007_00000_0000000001_00000?(+) to d0000000000_00000000000000000007_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796643Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000008_00000_0000000001_00000?(+) to d0000000000_00000000000000000008_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796645Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000009_00000_0000000001_00000?(+) to d0000000000_00000000000000000009_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796649Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000010_00000_0000000001_00000?(+) to d0000000000_00000000000000000010_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796651Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000011_00000_0000000001_00000?(+) to d0000000000_00000000000000000011_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796653Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000012_00000_0000000001_00000?(+) to d0000000000_00000000000000000012_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796655Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000013_00000_0000000001_00000?(+) to d0000000000_00000000000000000013_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796657Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000014_00000_0000000001_00000?(+) to d0000000000_00000000000000000014_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796659Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000015_00000_0000000001_00000?(+) to d0000000000_00000000000000000015_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796662Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000016_00000_0000000001_00000?(+) to d0000000000_00000000000000000016_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796664Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000017_00000_0000000001_00000?(+) to d0000000000_00000000000000000017_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796666Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000018_00000_0000000001_00000?(+) to d0000000000_00000000000000000018_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796668Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000019_00000_0000000001_00000?(+) to d0000000000_00000000000000000019_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796670Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000020_00000_0000000001_00000?(+) to d0000000000_00000000000000000020_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796673Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000021_00000_0000000001_00000?(+) to d0000000000_00000000000000000021_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796676Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000022_00000_0000000001_00000?(+) to d0000000000_00000000000000000022_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796678Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000023_00000_0000000001_00000?(+) to d0000000000_00000000000000000023_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796680Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000024_00000_0000000001_00000?(+) to d0000000000_00000000000000000024_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796684Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000025_00000_0000000001_00000?(+) to d0000000000_00000000000000000025_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796686Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000026_00000_0000000001_00000?(+) to d0000000000_00000000000000000026_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796688Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000027_00000_0000000001_00000?(+) to d0000000000_00000000000000000027_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796690Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000028_00000_0000000001_00000?(+) to d0000000000_00000000000000000028_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796694Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000029_00000_0000000001_00000?(+) to d0000000000_00000000000000000029_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796696Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000030_00000_0000000001_00000?(+) to d0000000000_00000000000000000030_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796699Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000031_00000_0000000001_00000?(+) to d0000000000_00000000000000000031_00000_0000000001_00000?(+)
2025-11-28T18:22:31.796816Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.797220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077531:3450] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 32
    Offset: 32
    WriteTimestampMS: 1764354151784
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 11
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.798003Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.797808Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 0 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797820Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 1 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797828Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 2 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 3 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797837Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 4 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797841Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 5 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797845Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 6 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797850Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 7 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797855Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 8 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797859Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 9 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797864Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 10 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797868Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 11 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797872Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 12 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797876Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 13 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797881Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 14 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 15 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797890Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 16 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797894Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 17 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797899Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 18 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797903Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 19 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797907Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 20 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797911Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 21 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797916Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 22 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797920Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 23 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797924Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 24 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797928Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 25 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797933Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 26 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797937Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 27 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797941Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 28 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797945Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 29 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797949Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 30 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797953Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 31 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.797970Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.797974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.797975Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.797978Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.797980Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.797982Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.797985Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.797985Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.797988Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.797992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.797995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.797997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.797999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.798000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.798004Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.798006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.798008Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.798012Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.798016Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.798019Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.798022Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.798028Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.798031Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.798034Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.798037Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.798039Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.798041Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.798043Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.798045Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.798047Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.798049Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.798051Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.798053Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.798055Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.798057Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.798061Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.798063Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.798065Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.798067Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.798069Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.798480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077537:3454], now have 1 active actors on pipe
2025-11-28T18:22:31.798635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.798643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.798670Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|bd1e4782-6da34ae0-6bf4e7b5-3875a1f1_33 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.798676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.798679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.798682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.798733Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.798684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.798687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.798694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.798695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.798698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.798706Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.798714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.798939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077537:3454] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|bd1e4782-6da34ae0-6bf4e7b5-3875a1f1_33"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|bd1e4782-6da34ae0-6bf4e7b5-3875a1f1_33"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 33
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.801002Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.801418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077541:3456], now have 1 active actors on pipe
2025-11-28T18:22:31.801759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.801773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.801795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 33 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.801806Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.801818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.801822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.801832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.801834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.801838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.802781Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 33 partNo 0
2025-11-28T18:22:31.802832Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 33 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 262214 count 1 nextOffset 34 batches 1
2025-11-28T18:22:31.802848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.802851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.802854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.802955Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 33,1 HeadOffset 33 endOffset 33 curOffset 34 d0000000000_00000000000000000033_00000_0000000001_00000? size 262204 WTime 1764354151802
2025-11-28T18:22:31.803013Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.803033Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 33 partNo 0 count 1 size 262204
2025-11-28T18:22:31.808410Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 33 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.808443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.808462Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.808463Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 33 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.808471Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.808480Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 33, partNo: 0, Offset: 33 is stored on disk
2025-11-28T18:22:31.808510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.808512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.808514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.808517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.808519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.808526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.808537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.808608Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 33
    Offset: 33
    WriteTimestampMS: 1764354151801
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
2025-11-28T18:22:31.809187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077541:3456] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.809962Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.810272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077546:3459], now have 1 active actors on pipe
2025-11-28T18:22:31.810417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.810429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.810474Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|65a1f100-4b1107f8-792c1c1a-3e616ad1_34 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.810482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.810485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.810489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.810491Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.810493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.810500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.810502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.810504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.810516Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.810528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.810648Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|65a1f100-4b1107f8-792c1c1a-3e616ad1_34"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
2025-11-28T18:22:31.810973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077546:3459] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|65a1f100-4b1107f8-792c1c1a-3e616ad1_34"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 34
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.812872Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.814320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077549:3460], now have 1 active actors on pipe
2025-11-28T18:22:31.814414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.814419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.814436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 34 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.814445Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.814457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.814481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.814489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.814491Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.814495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.815435Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 34 partNo 0
2025-11-28T18:22:31.815483Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 34 partNo 0 FormedBlobsCount 0 NewHead: Offset 34 PartNo 0 PackedSize 262214 count 1 nextOffset 35 batches 1
2025-11-28T18:22:31.815491Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.815494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.815497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.815586Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 34,1 HeadOffset 34 endOffset 34 curOffset 35 d0000000000_00000000000000000034_00000_0000000001_00000? size 262204 WTime 1764354151815
2025-11-28T18:22:31.815629Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.815641Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 34 partNo 0 count 1 size 262204
2025-11-28T18:22:31.818082Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 34 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.818105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.818124Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.818133Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.818142Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 34, partNo: 0, Offset: 34 is stored on disk
2025-11-28T18:22:31.818170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.818179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.818183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.818185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.818187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.818189Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 34 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.818193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.818203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.818281Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 34
    Offset: 34
    WriteTimestampMS: 1764354151814
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.818773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077549:3460] destroyed
2025-11-28T18:22:31.819365Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.819747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077553:3462], now have 1 active actors on pipe
2025-11-28T18:22:31.819928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.819938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.819970Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|87d761c6-f5bab181-ac8735cb-ad0d9b6d_35 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.819980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.819985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.819992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.819994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.819998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.820020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.820023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.820028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.820042Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.820056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.820231Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.820553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077553:3462] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|87d761c6-f5bab181-ac8735cb-ad0d9b6d_35"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|87d761c6-f5bab181-ac8735cb-ad0d9b6d_35"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 35
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.824973Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.825477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077556:3463], now have 1 active actors on pipe
2025-11-28T18:22:31.825995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.826009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.826029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 35 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.826041Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.826056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.826060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.826070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.826073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.826077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.827029Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 35 partNo 0
2025-11-28T18:22:31.827079Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 35 partNo 0 FormedBlobsCount 0 NewHead: Offset 35 PartNo 0 PackedSize 262214 count 1 nextOffset 36 batches 1
2025-11-28T18:22:31.827095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.827098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.827104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.827215Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 35,1 HeadOffset 35 endOffset 35 curOffset 36 d0000000000_00000000000000000035_00000_0000000001_00000? size 262204 WTime 1764354151827
2025-11-28T18:22:31.827271Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.827290Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 35 partNo 0 count 1 size 262204
2025-11-28T18:22:31.830718Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 35 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.830740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.830763Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.830773Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.830784Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 35, partNo: 0, Offset: 35 is stored on disk
2025-11-28T18:22:31.830824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.830827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.830829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.830831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.830833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.830841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.830853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.830906Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 35 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.831052Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.831457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077556:3463] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 35
    Offset: 35
    WriteTimestampMS: 1764354151825
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.835433Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.835980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077560:3465], now have 1 active actors on pipe
2025-11-28T18:22:31.836006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.836017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.836051Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ff2e4b8a-ce21564d-9df0d264-2eb16d16_36 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.836067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.836076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.836083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.836089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.836094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.836104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.836112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.836115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.836132Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.836153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.836244Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.836555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077560:3465] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ff2e4b8a-ce21564d-9df0d264-2eb16d16_36"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ff2e4b8a-ce21564d-9df0d264-2eb16d16_36"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 36
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.838915Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.839459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077563:3466], now have 1 active actors on pipe
2025-11-28T18:22:31.839856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.839872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.839894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 36 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.839908Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.839923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.839932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.839943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.839947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.839953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.841021Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 36 partNo 0
2025-11-28T18:22:31.841070Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 36 partNo 0 FormedBlobsCount 0 NewHead: Offset 36 PartNo 0 PackedSize 262214 count 1 nextOffset 37 batches 1
2025-11-28T18:22:31.841080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.841084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.841087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.841185Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 36,1 HeadOffset 36 endOffset 36 curOffset 37 d0000000000_00000000000000000036_00000_0000000001_00000? size 262204 WTime 1764354151840
2025-11-28T18:22:31.841234Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.841247Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 36 partNo 0 count 1 size 262204
2025-11-28T18:22:31.843759Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 36 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.843781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.843803Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.843815Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.843826Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 36, partNo: 0, Offset: 36 is stored on disk
2025-11-28T18:22:31.843866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.843872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.843874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.843877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.843879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.843887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.843901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.844019Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 36 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.844136Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.844549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 36
    Offset: 36
    WriteTimestampMS: 1764354151839
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:31.844555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.844558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.844560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.844562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.844575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077563:3466] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.845217Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.845657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077567:3468], now have 1 active actors on pipe
2025-11-28T18:22:31.845844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.845853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.845887Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|bc95f175-c29692f8-7ac50224-7392a1dd_37 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.845895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.845899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.845904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.845906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.845910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.845919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.845921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.845924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.845935Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.845947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.846013Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.846411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077567:3468] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|bc95f175-c29692f8-7ac50224-7392a1dd_37"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|bc95f175-c29692f8-7ac50224-7392a1dd_37"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 37
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.848734Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.849189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077570:3469], now have 1 active actors on pipe
2025-11-28T18:22:31.849626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.849643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.849665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 37 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.849678Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.849692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.849697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.849723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.849725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.849730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.850767Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 37 partNo 0
2025-11-28T18:22:31.850820Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 37 partNo 0 FormedBlobsCount 0 NewHead: Offset 37 PartNo 0 PackedSize 262214 count 1 nextOffset 38 batches 1
2025-11-28T18:22:31.850832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.850837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.850841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.850950Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 37,1 HeadOffset 37 endOffset 37 curOffset 38 d0000000000_00000000000000000037_00000_0000000001_00000? size 262204 WTime 1764354151850
2025-11-28T18:22:31.851002Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.851015Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 37 partNo 0 count 1 size 262204
2025-11-28T18:22:31.853508Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 37 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.853533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.853549Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 37 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.853555Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.853565Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.853575Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 37, partNo: 0, Offset: 37 is stored on disk
2025-11-28T18:22:31.853611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.853614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.853616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.853621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.853623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.853632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.853646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.853735Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.854278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077570:3469] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 37
    Offset: 37
    WriteTimestampMS: 1764354151849
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.854937Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.855383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077574:3471], now have 1 active actors on pipe
2025-11-28T18:22:31.855533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.855542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.855578Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ea470cef-358a210-dc21fa8d-c533eda3_38 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.855586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.855590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.855596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.855598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.855602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.855612Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.855614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.855616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.855629Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.855643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.855695Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.855941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077574:3471] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ea470cef-358a210-dc21fa8d-c533eda3_38"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ea470cef-358a210-dc21fa8d-c533eda3_38"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 38
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.858193Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.858679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077577:3472], now have 1 active actors on pipe
2025-11-28T18:22:31.859161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.859175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.859197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 38 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.859207Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.859219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.859222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.859234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.859236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.859240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.860241Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 38 partNo 0
2025-11-28T18:22:31.860278Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 38 partNo 0 FormedBlobsCount 0 NewHead: Offset 38 PartNo 0 PackedSize 262214 count 1 nextOffset 39 batches 1
2025-11-28T18:22:31.860287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.860289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.860292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.860383Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 38,1 HeadOffset 38 endOffset 38 curOffset 39 d0000000000_00000000000000000038_00000_0000000001_00000? size 262204 WTime 1764354151859
2025-11-28T18:22:31.860433Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.860448Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 38 partNo 0 count 1 size 262204
2025-11-28T18:22:31.862919Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 38 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.862950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.862972Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.862973Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 38 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.862983Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.862993Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 38, partNo: 0, Offset: 38 is stored on disk
2025-11-28T18:22:31.863024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.863027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.863028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.863031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.863033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.863042Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.863053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.863152Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.863545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077577:3472] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 38
    Offset: 38
    WriteTimestampMS: 1764354151858
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.864332Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.864909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077581:3474], now have 1 active actors on pipe
2025-11-28T18:22:31.865020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.865029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.865064Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9bc305aa-50d5d1cc-e8035ade-e5106d44_39 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.865074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.865079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.865084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.865086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.865090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.865098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.865101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.865104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.865117Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.865129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.865189Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9bc305aa-50d5d1cc-e8035ade-e5106d44_39"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
2025-11-28T18:22:31.865491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077581:3474] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9bc305aa-50d5d1cc-e8035ade-e5106d44_39"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 39
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.867677Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.868120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077584:3475], now have 1 active actors on pipe
2025-11-28T18:22:31.868478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.868492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.868513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 39 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.868523Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.868535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.868540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.868551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.868554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.868557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.869603Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 39 partNo 0
2025-11-28T18:22:31.869653Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 39 partNo 0 FormedBlobsCount 0 NewHead: Offset 39 PartNo 0 PackedSize 262214 count 1 nextOffset 40 batches 1
2025-11-28T18:22:31.869664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.869667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.869670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.869776Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 39,1 HeadOffset 39 endOffset 39 curOffset 40 d0000000000_00000000000000000039_00000_0000000001_00000? size 262204 WTime 1764354151869
2025-11-28T18:22:31.869824Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.869846Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 39 partNo 0 count 1 size 262204
2025-11-28T18:22:31.872030Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 39 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.872059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.872081Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 39 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.872082Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.872091Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.872100Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 39, partNo: 0, Offset: 39 is stored on disk
2025-11-28T18:22:31.872135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.872138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.872140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.872143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.872145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.872152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.872163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.872239Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.872808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077584:3475] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 39
    Offset: 39
    WriteTimestampMS: 1764354151868
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.873547Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.873982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077588:3477], now have 1 active actors on pipe
2025-11-28T18:22:31.874074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.874083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.874116Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|50f24692-4c8a7f05-eb905728-9e4f6ce7_40 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.874124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.874127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.874132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.874134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.874138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.874147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.874149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.874152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.874166Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.874179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.874238Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|50f24692-4c8a7f05-eb905728-9e4f6ce7_40"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
2025-11-28T18:22:31.874693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077588:3477] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|50f24692-4c8a7f05-eb905728-9e4f6ce7_40"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 40
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.876669Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.877103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077591:3478], now have 1 active actors on pipe
2025-11-28T18:22:31.877467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.877482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.877505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 40 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.877517Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.877529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.877533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.877545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.877548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.877554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.878610Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 40 partNo 0
2025-11-28T18:22:31.878667Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 40 partNo 0 FormedBlobsCount 0 NewHead: Offset 40 PartNo 0 PackedSize 262214 count 1 nextOffset 41 batches 1
2025-11-28T18:22:31.878685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.878689Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.878692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.878801Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 40,1 HeadOffset 40 endOffset 40 curOffset 41 d0000000000_00000000000000000040_00000_0000000001_00000? size 262204 WTime 1764354151878
2025-11-28T18:22:31.878864Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.878876Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 40 partNo 0 count 1 size 262204
2025-11-28T18:22:31.883363Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 40 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.883391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.883413Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.883418Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 40 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.883424Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.883435Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 40, partNo: 0, Offset: 40 is stored on disk
2025-11-28T18:22:31.883473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.883482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.883483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.883485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.883487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.883497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.883514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.883608Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.884069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077591:3478] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 40
    Offset: 40
    WriteTimestampMS: 1764354151877
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.884764Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.885249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077595:3480], now have 1 active actors on pipe
2025-11-28T18:22:31.885426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.885440Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.885478Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|cb3c8ce2-35cdad01-d4686bbf-aeac2fa2_41 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.885492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.885500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.885505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.885514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.885519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.885532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.885539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.885541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.885556Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.885572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.885630Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.885933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077595:3480] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|cb3c8ce2-35cdad01-d4686bbf-aeac2fa2_41"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|cb3c8ce2-35cdad01-d4686bbf-aeac2fa2_41"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 41
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.887841Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.888528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077598:3481], now have 1 active actors on pipe
2025-11-28T18:22:31.888750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.888765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.888786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 41 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.888801Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.888818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.888826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.888836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.888842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.888846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.889884Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 41 partNo 0
2025-11-28T18:22:31.889928Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 41 partNo 0 FormedBlobsCount 0 NewHead: Offset 41 PartNo 0 PackedSize 262214 count 1 nextOffset 42 batches 1
2025-11-28T18:22:31.889938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.889944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.889949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.890038Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 41,1 HeadOffset 41 endOffset 41 curOffset 42 d0000000000_00000000000000000041_00000_0000000001_00000? size 262204 WTime 1764354151889
2025-11-28T18:22:31.890089Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.890110Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 41 partNo 0 count 1 size 262204
2025-11-28T18:22:31.892474Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 41 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.892500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.892518Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 41 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.892520Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.892529Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.892540Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 41, partNo: 0, Offset: 41 is stored on disk
2025-11-28T18:22:31.892574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.892583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.892585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.892588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.892590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.892597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.892614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.892708Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 41
    Offset: 41
    WriteTimestampMS: 1764354151888
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:31.893087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077598:3481] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.893644Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.894063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077602:3483], now have 1 active actors on pipe
2025-11-28T18:22:31.894178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.894190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.894222Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9e9c66f3-7233d335-79768a5a-466f902_42 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.894234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.894242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.894246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.894253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.894256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.894269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.894272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.894280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.894292Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.894310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.894381Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9e9c66f3-7233d335-79768a5a-466f902_42"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9e9c66f3-7233d335-79768a5a-466f902_42"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 42
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.895089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077602:3483] destroyed
2025-11-28T18:22:31.896367Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.896807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077605:3484], now have 1 active actors on pipe
2025-11-28T18:22:31.897204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.897218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.897240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 42 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.897255Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.897272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.897279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.897289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.897295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.897300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.898178Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 42 partNo 0
2025-11-28T18:22:31.898226Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 42 partNo 0 FormedBlobsCount 0 NewHead: Offset 42 PartNo 0 PackedSize 262214 count 1 nextOffset 43 batches 1
2025-11-28T18:22:31.898239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.898246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.898250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.898341Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 42,1 HeadOffset 42 endOffset 42 curOffset 43 d0000000000_00000000000000000042_00000_0000000001_00000? size 262204 WTime 1764354151897
2025-11-28T18:22:31.898393Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.898408Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 42 partNo 0 count 1 size 262204
2025-11-28T18:22:31.900930Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 42 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.900961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.900982Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 42 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.900985Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.900996Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.901006Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 42, partNo: 0, Offset: 42 is stored on disk
2025-11-28T18:22:31.901040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.901048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.901050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.901052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.901054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.901062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.901074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.901179Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 42
    Offset: 42
    WriteTimestampMS: 1764354151896
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:31.901509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077605:3484] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.902021Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.902382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077611:3488], now have 1 active actors on pipe
2025-11-28T18:22:31.902496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.902507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.902540Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|7aa910bd-6f757c57-17a2757b-41a49203_43 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.902554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.902561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.902567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.902568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.902571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.902583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.902590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.902592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.902604Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.902618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.902676Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|7aa910bd-6f757c57-17a2757b-41a49203_43"
    Status: Active
    SeqNo: 0
  }
}
2025-11-28T18:22:31.902921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077611:3488] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|7aa910bd-6f757c57-17a2757b-41a49203_43"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 43
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.904486Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.904875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077614:3489], now have 1 active actors on pipe
2025-11-28T18:22:31.905282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.905296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.905317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 43 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.905329Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.905341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.905350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.905360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.905362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.905366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.906264Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 43 partNo 0
2025-11-28T18:22:31.906310Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 43 partNo 0 FormedBlobsCount 0 NewHead: Offset 43 PartNo 0 PackedSize 262214 count 1 nextOffset 44 batches 1
2025-11-28T18:22:31.906323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.906330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.906334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.906424Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 43,1 HeadOffset 43 endOffset 43 curOffset 44 d0000000000_00000000000000000043_00000_0000000001_00000? size 262204 WTime 1764354151906
2025-11-28T18:22:31.906498Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.906514Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 43 partNo 0 count 1 size 262204
2025-11-28T18:22:31.908588Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 43 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.908608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.908625Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 43 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.908630Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.908640Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.908650Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 43, partNo: 0, Offset: 43 is stored on disk
2025-11-28T18:22:31.908691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.908698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.908699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.908702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.908704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.908711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.908731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.908829Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.909170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077614:3489] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 43
    Offset: 43
    WriteTimestampMS: 1764354151905
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.909841Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.910268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077618:3491], now have 1 active actors on pipe
2025-11-28T18:22:31.910404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.910416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.910449Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|5d06eea3-de7f1ef4-a12b97f4-cf5f53b6_44 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.910498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.910501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.910506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.910508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.910512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.910521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.910529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.910531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.910541Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.910553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.910611Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.910846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077618:3491] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|5d06eea3-de7f1ef4-a12b97f4-cf5f53b6_44"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|5d06eea3-de7f1ef4-a12b97f4-cf5f53b6_44"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 44
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.912883Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.913335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077621:3492], now have 1 active actors on pipe
2025-11-28T18:22:31.913821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.913833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.913865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 44 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.913883Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.913902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.913905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.913914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.913916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.913919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.914521Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 44 partNo 0
2025-11-28T18:22:31.914562Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 44 partNo 0 FormedBlobsCount 0 NewHead: Offset 44 PartNo 0 PackedSize 262214 count 1 nextOffset 45 batches 1
2025-11-28T18:22:31.914578Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.914584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.914586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.914676Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 44,1 HeadOffset 44 endOffset 44 curOffset 45 d0000000000_00000000000000000044_00000_0000000001_00000? size 262204 WTime 1764354151914
2025-11-28T18:22:31.914728Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.914745Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 44 partNo 0 count 1 size 262204
2025-11-28T18:22:31.916994Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 44 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.917016Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.917036Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 44 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.917038Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.917047Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.917054Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 44, partNo: 0, Offset: 44 is stored on disk
2025-11-28T18:22:31.917087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.917090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.917091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.917093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.917094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.917102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.917113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.917199Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 44
    Offset: 44
    WriteTimestampMS: 1764354151913
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:31.917569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077621:3492] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.918263Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.918660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077625:3494], now have 1 active actors on pipe
2025-11-28T18:22:31.918754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.918766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.918799Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f6c700ec-d749d74a-192057b9-16b5d5d3_45 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.918811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.918814Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.918818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.918820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.918824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.918833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.918835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.918839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.918853Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.918877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.918932Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.919188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077625:3494] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f6c700ec-d749d74a-192057b9-16b5d5d3_45"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f6c700ec-d749d74a-192057b9-16b5d5d3_45"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 45
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.921164Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.921564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077628:3495], now have 1 active actors on pipe
2025-11-28T18:22:31.921980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.921994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.922014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 45 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.922026Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.922041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.922047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.922056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.922057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.922061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.922994Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 45 partNo 0
2025-11-28T18:22:31.923038Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 45 partNo 0 FormedBlobsCount 0 NewHead: Offset 45 PartNo 0 PackedSize 262214 count 1 nextOffset 46 batches 1
2025-11-28T18:22:31.923051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.923054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.923057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.923146Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 45,1 HeadOffset 45 endOffset 45 curOffset 46 d0000000000_00000000000000000045_00000_0000000001_00000? size 262204 WTime 1764354151922
2025-11-28T18:22:31.923194Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.923209Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 45 partNo 0 count 1 size 262204
2025-11-28T18:22:31.925240Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 45 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.925261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.925278Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 45 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.925280Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.925288Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.925297Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 45, partNo: 0, Offset: 45 is stored on disk
2025-11-28T18:22:31.925324Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.925331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.925334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.925336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.925337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.925344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.925354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.925427Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.925767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077628:3495] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 45
    Offset: 45
    WriteTimestampMS: 1764354151921
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.926673Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.927126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077632:3497], now have 1 active actors on pipe
2025-11-28T18:22:31.927249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.927264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.927300Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|582dbe5b-83232e82-49a32df2-54f46e1a_46 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.927313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.927317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.927327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.927329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.927333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.927344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.927351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.927354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.927368Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.927386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.927442Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.927711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077632:3497] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|582dbe5b-83232e82-49a32df2-54f46e1a_46"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|582dbe5b-83232e82-49a32df2-54f46e1a_46"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 46
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.929998Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.930444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077635:3498], now have 1 active actors on pipe
2025-11-28T18:22:31.930859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.930876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.930897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 46 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.930913Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.930925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.930929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.930940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.930943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.930948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.931902Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 46 partNo 0
2025-11-28T18:22:31.931957Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 46 partNo 0 FormedBlobsCount 0 NewHead: Offset 46 PartNo 0 PackedSize 262214 count 1 nextOffset 47 batches 1
2025-11-28T18:22:31.931974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.931976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.931981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.932078Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 46,1 HeadOffset 46 endOffset 46 curOffset 47 d0000000000_00000000000000000046_00000_0000000001_00000? size 262204 WTime 1764354151931
2025-11-28T18:22:31.932134Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.932152Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 46 partNo 0 count 1 size 262204
2025-11-28T18:22:31.934108Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 46 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.934132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.934153Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.934156Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 46 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.934163Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.934174Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 46, partNo: 0, Offset: 46 is stored on disk
2025-11-28T18:22:31.934206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.934215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.934218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.934220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.934222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.934230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.934243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.934321Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 46
    Offset: 46
    WriteTimestampMS: 1764354151930
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:31.934870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077635:3498] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.935499Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.935874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077639:3500], now have 1 active actors on pipe
2025-11-28T18:22:31.935987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.936000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.936036Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|fbff5176-e6b573c2-1b41356d-58f899e2_47 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.936045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.936053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.936058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.936060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.936064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.936072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.936075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.936078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.936092Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.936110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.936165Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|fbff5176-e6b573c2-1b41356d-58f899e2_47"
    Status: Active
    SeqNo: 0
  }
}
2025-11-28T18:22:31.936431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077639:3500] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|fbff5176-e6b573c2-1b41356d-58f899e2_47"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 47
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.938091Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.938508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077642:3501], now have 1 active actors on pipe
2025-11-28T18:22:31.938875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.938896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.938920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 47 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.938938Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.938958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.938970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.938983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.938985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.938991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.939945Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 47 partNo 0
2025-11-28T18:22:31.939994Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 47 partNo 0 FormedBlobsCount 0 NewHead: Offset 47 PartNo 0 PackedSize 262214 count 1 nextOffset 48 batches 1
2025-11-28T18:22:31.940010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.940013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.940017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.940118Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 47,1 HeadOffset 47 endOffset 47 curOffset 48 d0000000000_00000000000000000047_00000_0000000001_00000? size 262204 WTime 1764354151939
2025-11-28T18:22:31.940172Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.940190Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 47 partNo 0 count 1 size 262204
2025-11-28T18:22:31.942796Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 47 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.942826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.942846Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.942855Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 47 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.942859Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.942870Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 47, partNo: 0, Offset: 47 is stored on disk
2025-11-28T18:22:31.942906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.942915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.942917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.942920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.942924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.942932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.942945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.943036Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.943351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077642:3501] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 47
    Offset: 47
    WriteTimestampMS: 1764354151938
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.944098Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.944531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077646:3503], now have 1 active actors on pipe
2025-11-28T18:22:31.944564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.944567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.944569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.944572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.944574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.944594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.944607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.944642Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|6be7338e-5c6044a8-96e04077-8a3ff4d8_48 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.944651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.944654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.944658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.944659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.944662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.944671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.944674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.944676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.944686Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.944699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
2025-11-28T18:22:31.945039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077646:3503] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|6be7338e-5c6044a8-96e04077-8a3ff4d8_48"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
2025-11-28T18:22:31.944815Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|6be7338e-5c6044a8-96e04077-8a3ff4d8_48"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 48
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.947269Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.947606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077649:3504], now have 1 active actors on pipe
2025-11-28T18:22:31.947975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.947987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.948009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 48 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.948026Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.948040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.948045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.948054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.948060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.948064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.949100Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 48 partNo 0
2025-11-28T18:22:31.949148Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 48 partNo 0 FormedBlobsCount 0 NewHead: Offset 48 PartNo 0 PackedSize 262214 count 1 nextOffset 49 batches 1
2025-11-28T18:22:31.949164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.949170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.949173Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.949264Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 48,1 HeadOffset 48 endOffset 48 curOffset 49 d0000000000_00000000000000000048_00000_0000000001_00000? size 262204 WTime 1764354151949
2025-11-28T18:22:31.949318Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.949338Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 48 partNo 0 count 1 size 262204
2025-11-28T18:22:31.951596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 48 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.951639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.951651Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 48 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.951661Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.951671Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.951680Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 48, partNo: 0, Offset: 48 is stored on disk
2025-11-28T18:22:31.951721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.951732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.951734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.951737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.951739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.951748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.951759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.951859Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.952184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077649:3504] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 48
    Offset: 48
    WriteTimestampMS: 1764354151948
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.952993Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.953411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077657:3510], now have 1 active actors on pipe
2025-11-28T18:22:31.953497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.953511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.953549Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|55e35f5a-b9448cda-6ade4e03-8d8c2eb4_49 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.953564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.953567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.953572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.953575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.953578Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.953588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.953594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.953597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.953609Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.953626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.953681Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|55e35f5a-b9448cda-6ade4e03-8d8c2eb4_49"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
2025-11-28T18:22:31.954363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077657:3510] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|55e35f5a-b9448cda-6ade4e03-8d8c2eb4_49"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 49
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.956134Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.956537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077660:3511], now have 1 active actors on pipe
2025-11-28T18:22:31.956942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.956956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.956977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 49 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.956993Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.957005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.957009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.957017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.957019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.957024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.958021Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 49 partNo 0
2025-11-28T18:22:31.958067Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 49 partNo 0 FormedBlobsCount 0 NewHead: Offset 49 PartNo 0 PackedSize 262214 count 1 nextOffset 50 batches 1
2025-11-28T18:22:31.958076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.958081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.958084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.958174Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 49,1 HeadOffset 49 endOffset 49 curOffset 50 d0000000000_00000000000000000049_00000_0000000001_00000? size 262204 WTime 1764354151957
2025-11-28T18:22:31.958220Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.958233Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 49 partNo 0 count 1 size 262204
2025-11-28T18:22:31.960505Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 49 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.960532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.960551Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.960554Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 49 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.960565Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.960574Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 49, partNo: 0, Offset: 49 is stored on disk
2025-11-28T18:22:31.960605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.960615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.960617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.960619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.960621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.960629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.960644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.960744Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.961032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077660:3511] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 49
    Offset: 49
    WriteTimestampMS: 1764354151956
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.961612Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.962147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077664:3513], now have 1 active actors on pipe
2025-11-28T18:22:31.962172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.962178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.962209Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|32822f03-dabbd26e-474c2156-d45bcca8_50 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.962220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.962223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.962227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.962229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.962232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.962241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.962244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.962247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.962256Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.962270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.962320Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.962546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077664:3513] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|32822f03-dabbd26e-474c2156-d45bcca8_50"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|32822f03-dabbd26e-474c2156-d45bcca8_50"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 50
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.964638Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.965064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077667:3514], now have 1 active actors on pipe
2025-11-28T18:22:31.965412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.965427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.965448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 50 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.965463Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.965475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.965483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.965493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.965495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.965499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.966663Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 50 partNo 0
2025-11-28T18:22:31.966708Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 50 partNo 0 FormedBlobsCount 0 NewHead: Offset 50 PartNo 0 PackedSize 262214 count 1 nextOffset 51 batches 1
2025-11-28T18:22:31.966722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.966729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.966733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.966834Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 50,1 HeadOffset 50 endOffset 50 curOffset 51 d0000000000_00000000000000000050_00000_0000000001_00000? size 262204 WTime 1764354151966
2025-11-28T18:22:31.966883Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.966900Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 50 partNo 0 count 1 size 262204
2025-11-28T18:22:31.968977Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 50 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.969000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.969015Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 50 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.969019Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.969028Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.969038Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 50, partNo: 0, Offset: 50 is stored on disk
2025-11-28T18:22:31.969068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.969076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.969078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.969080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.969083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.969090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.969106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.969210Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.969586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077667:3514] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 50
    Offset: 50
    WriteTimestampMS: 1764354151965
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.970502Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.970961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077670:3515], now have 1 active actors on pipe
2025-11-28T18:22:31.971066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.971074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.971109Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|615281bd-886300e4-f58e9c16-b7484380_51 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.971117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.971119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.971123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.971125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.971128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.971138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.971141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.971146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.971159Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.971170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.971226Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.971457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077670:3515] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|615281bd-886300e4-f58e9c16-b7484380_51"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|615281bd-886300e4-f58e9c16-b7484380_51"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 51
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.975264Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.976074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077673:3516], now have 1 active actors on pipe
2025-11-28T18:22:31.976486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.976501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.976528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 51 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.976546Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.976563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.976567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.976580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.976583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.976588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.977602Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 51 partNo 0
2025-11-28T18:22:31.977648Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 51 partNo 0 FormedBlobsCount 0 NewHead: Offset 51 PartNo 0 PackedSize 262214 count 1 nextOffset 52 batches 1
2025-11-28T18:22:31.977664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.977667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.977670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.977772Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 51,1 HeadOffset 51 endOffset 51 curOffset 52 d0000000000_00000000000000000051_00000_0000000001_00000? size 262204 WTime 1764354151977
2025-11-28T18:22:31.977825Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.977843Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 51 partNo 0 count 1 size 262204
2025-11-28T18:22:31.980196Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 51 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.980225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.980238Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 51 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.980245Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.980255Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.980263Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 51, partNo: 0, Offset: 51 is stored on disk
2025-11-28T18:22:31.980297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.980301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.980303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.980307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.980308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.980316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.980333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.980422Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.980813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077673:3516] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 51
    Offset: 51
    WriteTimestampMS: 1764354151976
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.981717Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.982136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077677:3518], now have 1 active actors on pipe
2025-11-28T18:22:31.982235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.982243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.982278Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b383a58e-e98ef973-99823fbe-3c611f5f_52 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.982287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.982290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.982295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.982297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.982301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.982310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.982313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.982316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.982327Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.982338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.982383Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.982630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077677:3518] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b383a58e-e98ef973-99823fbe-3c611f5f_52"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b383a58e-e98ef973-99823fbe-3c611f5f_52"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 52
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.984926Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.985320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077680:3519], now have 1 active actors on pipe
2025-11-28T18:22:31.985664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.985676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.985699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 52 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.985710Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.985724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.985729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.985738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.985741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.985745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.986966Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 52 partNo 0
2025-11-28T18:22:31.987005Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 52 partNo 0 FormedBlobsCount 0 NewHead: Offset 52 PartNo 0 PackedSize 262214 count 1 nextOffset 53 batches 1
2025-11-28T18:22:31.987013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.987016Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.987019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.987121Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 52,1 HeadOffset 52 endOffset 52 curOffset 53 d0000000000_00000000000000000052_00000_0000000001_00000? size 262204 WTime 1764354151986
2025-11-28T18:22:31.987173Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.987185Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 52 partNo 0 count 1 size 262204
2025-11-28T18:22:31.989399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 52 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.989422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.989443Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.989452Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.989454Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 52 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.989462Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 52, partNo: 0, Offset: 52 is stored on disk
2025-11-28T18:22:31.989490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.989493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.989495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.989496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.989498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.989506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.989515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.989584Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.989859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077680:3519] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 52
    Offset: 52
    WriteTimestampMS: 1764354151985
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.990582Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.991059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077684:3521], now have 1 active actors on pipe
2025-11-28T18:22:31.991171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.991180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.991213Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ee8fc93e-58b3d56d-eeac0a98-5629c9e9_53 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.991226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.991233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.991237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.991239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.991243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.991253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.991255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.991257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.991269Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.991282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.991330Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:31.991547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077684:3521] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ee8fc93e-58b3d56d-eeac0a98-5629c9e9_53"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ee8fc93e-58b3d56d-eeac0a98-5629c9e9_53"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 53
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:31.993623Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.994013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077687:3522], now have 1 active actors on pipe
2025-11-28T18:22:31.994326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.994346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.994369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 53 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:31.994385Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:31.994400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.994408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.994417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.994423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.994428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.995322Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 53 partNo 0
2025-11-28T18:22:31.995375Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 53 partNo 0 FormedBlobsCount 0 NewHead: Offset 53 PartNo 0 PackedSize 262214 count 1 nextOffset 54 batches 1
2025-11-28T18:22:31.995389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.995395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.995398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.995488Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 53,1 HeadOffset 53 endOffset 53 curOffset 54 d0000000000_00000000000000000053_00000_0000000001_00000? size 262204 WTime 1764354151995
2025-11-28T18:22:31.995537Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:31.995554Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 53 partNo 0 count 1 size 262204
2025-11-28T18:22:31.997945Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 53 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:31.997969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:31.997991Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:31.997998Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 53 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:31.998004Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:31.998014Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 53, partNo: 0, Offset: 53 is stored on disk
2025-11-28T18:22:31.998051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.998057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.998059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.998062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:31.998063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.998072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:31.998084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.998189Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:31.998560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077687:3522] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 53
    Offset: 53
    WriteTimestampMS: 1764354151994
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:31.999165Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:31.999582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843381054077691:3524], now have 1 active actors on pipe
2025-11-28T18:22:31.999685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:31.999695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:31.999741Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|7a1bd0bd-92a48c53-e1f6d610-b93c8e2a_54 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:31.999756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:31.999760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:31.999764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:31.999766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:31.999769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:31.999777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:31.999780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:31.999782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:31.999792Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:31.999802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:31.999846Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.000066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843381054077691:3524] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|7a1bd0bd-92a48c53-e1f6d610-b93c8e2a_54"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|7a1bd0bd-92a48c53-e1f6d610-b93c8e2a_54"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 54
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.001832Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.002231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349044990:3525], now have 1 active actors on pipe
2025-11-28T18:22:32.002624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.002636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.002655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 54 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.002662Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.002672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.002676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.002684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.002689Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.002692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.003312Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 54 partNo 0
2025-11-28T18:22:32.003345Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 54 partNo 0 FormedBlobsCount 0 NewHead: Offset 54 PartNo 0 PackedSize 262214 count 1 nextOffset 55 batches 1
2025-11-28T18:22:32.003354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.003356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.003359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.003421Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 54,1 HeadOffset 54 endOffset 54 curOffset 55 d0000000000_00000000000000000054_00000_0000000001_00000? size 262204 WTime 1764354152003
2025-11-28T18:22:32.003457Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.003474Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 54 partNo 0 count 1 size 262204
2025-11-28T18:22:32.005697Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 54 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.005727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.005744Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 54 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.005747Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.005756Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.005762Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 54, partNo: 0, Offset: 54 is stored on disk
2025-11-28T18:22:32.005784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.005790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.005791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.005793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.005794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.005799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.005808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.005895Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:32.006223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349044990:3525] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 54
    Offset: 54
    WriteTimestampMS: 1764354152002
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.006955Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.007344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349044994:3527], now have 1 active actors on pipe
2025-11-28T18:22:32.007439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.007449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.007477Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|250c789e-dc4571e6-7df6765a-461d6b6b_55 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.007487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.007489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.007493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.007495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.007497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.007507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.007509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.007511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.007519Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.007528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.007577Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:32.007795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349044994:3527] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|250c789e-dc4571e6-7df6765a-461d6b6b_55"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|250c789e-dc4571e6-7df6765a-461d6b6b_55"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 55
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.009829Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.010230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349044997:3528], now have 1 active actors on pipe
2025-11-28T18:22:32.010589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.010599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.010620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 55 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.010632Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.010643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.010648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.010663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.010667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.010671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.011245Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 55 partNo 0
2025-11-28T18:22:32.011275Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 55 partNo 0 FormedBlobsCount 0 NewHead: Offset 55 PartNo 0 PackedSize 262214 count 1 nextOffset 56 batches 1
2025-11-28T18:22:32.011285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.011287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.011289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.011351Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 55,1 HeadOffset 55 endOffset 55 curOffset 56 d0000000000_00000000000000000055_00000_0000000001_00000? size 262204 WTime 1764354152010
2025-11-28T18:22:32.011387Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.011399Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 55 partNo 0 count 1 size 262204
2025-11-28T18:22:32.013976Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 55 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.014000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.014018Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.014019Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 55 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.014025Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.014032Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 55, partNo: 0, Offset: 55 is stored on disk
2025-11-28T18:22:32.014060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.014066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.014067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.014069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.014070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.014077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.014086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.014173Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.014503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349044997:3528] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 55
    Offset: 55
    WriteTimestampMS: 1764354152010
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.015272Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.015616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045001:3530], now have 1 active actors on pipe
2025-11-28T18:22:32.015738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.015748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.015776Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|27162ba3-bb92d589-f1c84ad7-2a962a68_56 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.015785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.015788Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.015791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.015792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.015794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.015801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.015803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.015805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.015815Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.015824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.015886Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.016086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045001:3530] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|27162ba3-bb92d589-f1c84ad7-2a962a68_56"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|27162ba3-bb92d589-f1c84ad7-2a962a68_56"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 56
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.017859Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.018206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045004:3531], now have 1 active actors on pipe
2025-11-28T18:22:32.018516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.018528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.018548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 56 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.018558Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.018575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.018582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.018591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.018597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.018601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.019420Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 56 partNo 0
2025-11-28T18:22:32.019457Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 56 partNo 0 FormedBlobsCount 0 NewHead: Offset 56 PartNo 0 PackedSize 262214 count 1 nextOffset 57 batches 1
2025-11-28T18:22:32.019467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.019469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.019471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.019532Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 56,1 HeadOffset 56 endOffset 56 curOffset 57 d0000000000_00000000000000000056_00000_0000000001_00000? size 262204 WTime 1764354152019
2025-11-28T18:22:32.019578Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.019594Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 56 partNo 0 count 1 size 262204
2025-11-28T18:22:32.021540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 56 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.021556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.021572Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.021579Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.021585Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 56, partNo: 0, Offset: 56 is stored on disk
2025-11-28T18:22:32.021608Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 56 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.021613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.021615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.021616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.021617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.021619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.021623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.021631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.021675Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.021917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045004:3531] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 56
    Offset: 56
    WriteTimestampMS: 1764354152018
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.022523Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.022853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045008:3533], now have 1 active actors on pipe
2025-11-28T18:22:32.022920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.022927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.022953Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|e2565bcc-68d60910-74f23580-cc1548af_57 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.022962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.022964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.022968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.022969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.022971Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.022977Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.022979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.022980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.022988Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.022995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.023024Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.023167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045008:3533] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|e2565bcc-68d60910-74f23580-cc1548af_57"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|e2565bcc-68d60910-74f23580-cc1548af_57"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 57
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.025045Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.025359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045011:3534], now have 1 active actors on pipe
2025-11-28T18:22:32.025667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.025682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.025702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 57 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.025713Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.025729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.025739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.025749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.025753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.025756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.026622Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 57 partNo 0
2025-11-28T18:22:32.026669Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 57 partNo 0 FormedBlobsCount 0 NewHead: Offset 57 PartNo 0 PackedSize 262214 count 1 nextOffset 58 batches 1
2025-11-28T18:22:32.026677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.026680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.026682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.026767Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 57,1 HeadOffset 57 endOffset 57 curOffset 58 d0000000000_00000000000000000057_00000_0000000001_00000? size 262204 WTime 1764354152026
2025-11-28T18:22:32.026817Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.026832Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 57 partNo 0 count 1 size 262204
2025-11-28T18:22:32.028983Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 57 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.029003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.029023Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.029030Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 57 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.029031Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.029039Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 57, partNo: 0, Offset: 57 is stored on disk
2025-11-28T18:22:32.029066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.029072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.029074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.029076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.029079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.029086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.029095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.029163Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.029436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045011:3534] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 57
    Offset: 57
    WriteTimestampMS: 1764354152025
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.030091Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.030438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045015:3536], now have 1 active actors on pipe
2025-11-28T18:22:32.030552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.030564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.030600Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|8610f2bc-f8af5bd-2bf352cc-b08df65f_58 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.030611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.030614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.030618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.030620Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.030622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.030630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.030632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.030635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.030645Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.030659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.030709Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.030905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045015:3536] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|8610f2bc-f8af5bd-2bf352cc-b08df65f_58"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|8610f2bc-f8af5bd-2bf352cc-b08df65f_58"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 58
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.032650Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.033046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045018:3537], now have 1 active actors on pipe
2025-11-28T18:22:32.033434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.033447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.033465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 58 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.033473Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.033493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.033499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.033506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.033508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.033511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.034344Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 58 partNo 0
2025-11-28T18:22:32.034389Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 58 partNo 0 FormedBlobsCount 0 NewHead: Offset 58 PartNo 0 PackedSize 262214 count 1 nextOffset 59 batches 1
2025-11-28T18:22:32.034404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.034407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.034410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.034515Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 58,1 HeadOffset 58 endOffset 58 curOffset 59 d0000000000_00000000000000000058_00000_0000000001_00000? size 262204 WTime 1764354152034
2025-11-28T18:22:32.034561Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.034577Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 58 partNo 0 count 1 size 262204
2025-11-28T18:22:32.036930Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 58 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.036958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.036978Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.036988Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 58 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.036990Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.036998Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 58, partNo: 0, Offset: 58 is stored on disk
2025-11-28T18:22:32.037033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.037039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.037040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.037043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.037044Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.037051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.037062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.037168Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.037547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045018:3537] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 58
    Offset: 58
    WriteTimestampMS: 1764354152033
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.038285Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.038768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045022:3539], now have 1 active actors on pipe
2025-11-28T18:22:32.038830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.038840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.038876Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|53254922-46d4010f-4c7cba51-e46bb5b2_59 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.038890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.038896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.038900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.038902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.038907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.038918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.038925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.038927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.038940Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.038957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.039010Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.039238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045022:3539] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|53254922-46d4010f-4c7cba51-e46bb5b2_59"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|53254922-46d4010f-4c7cba51-e46bb5b2_59"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 59
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.041379Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.041828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045025:3540], now have 1 active actors on pipe
2025-11-28T18:22:32.042245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.042279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.042299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 59 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.042312Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.042325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.042333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.042345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.042351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.042354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.042977Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 59 partNo 0
2025-11-28T18:22:32.043014Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 59 partNo 0 FormedBlobsCount 0 NewHead: Offset 59 PartNo 0 PackedSize 262214 count 1 nextOffset 60 batches 1
2025-11-28T18:22:32.043022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.043025Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.043028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.043092Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 59,1 HeadOffset 59 endOffset 59 curOffset 60 d0000000000_00000000000000000059_00000_0000000001_00000? size 262204 WTime 1764354152042
2025-11-28T18:22:32.043132Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.043144Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 59 partNo 0 count 1 size 262204
2025-11-28T18:22:32.044860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-28T18:22:32.045693Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 59 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.045712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.045731Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.045739Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.045739Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 59 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.045748Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 59, partNo: 0, Offset: 59 is stored on disk
2025-11-28T18:22:32.045780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.045786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.045789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.045791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.045793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.045800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.045809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.045891Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 59
    Offset: 59
    WriteTimestampMS: 1764354152042
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:32.046225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045025:3540] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.046903Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.047278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045029:3542], now have 1 active actors on pipe
2025-11-28T18:22:32.047376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.047385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.047415Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|6240d636-b8af889f-ba15f40-ac6cdccb_60 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.047427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.047430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.047434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.047440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.047444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.047452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.047454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.047457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.047467Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.047480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.047591Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|6240d636-b8af889f-ba15f40-ac6cdccb_60"
    Status: Active
    SeqNo: 0
  }
}
2025-11-28T18:22:32.047844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045029:3542] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|6240d636-b8af889f-ba15f40-ac6cdccb_60"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 60
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.049591Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.050007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045032:3543], now have 1 active actors on pipe
2025-11-28T18:22:32.050407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.050426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.050449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 60 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.050473Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.050487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.050493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.050502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.050509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.050513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.051478Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 60 partNo 0
2025-11-28T18:22:32.051528Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 60 partNo 0 FormedBlobsCount 0 NewHead: Offset 60 PartNo 0 PackedSize 262214 count 1 nextOffset 61 batches 1
2025-11-28T18:22:32.051538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.051541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.051544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.051641Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 60,1 HeadOffset 60 endOffset 60 curOffset 61 d0000000000_00000000000000000060_00000_0000000001_00000? size 262204 WTime 1764354152051
2025-11-28T18:22:32.051691Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.051711Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 60 partNo 0 count 1 size 262204
2025-11-28T18:22:32.054040Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 60 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.054068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.054088Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.054096Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.054106Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 60 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.054108Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 60, partNo: 0, Offset: 60 is stored on disk
2025-11-28T18:22:32.054138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.054145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.054147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.054148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.054151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.054158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.054168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.054286Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:32.054606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045032:3543] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 60
    Offset: 60
    WriteTimestampMS: 1764354152050
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.055285Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.055641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045036:3545], now have 1 active actors on pipe
2025-11-28T18:22:32.055776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.055787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.055819Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|6e68859d-d80fb924-4fb20df4-e06d20d5_61 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.055830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.055833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.055838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.055840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.055843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.055852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.055854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.055857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.055867Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.055883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.055941Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:32.056141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045036:3545] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|6e68859d-d80fb924-4fb20df4-e06d20d5_61"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|6e68859d-d80fb924-4fb20df4-e06d20d5_61"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 61
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.058143Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.058516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045039:3546], now have 1 active actors on pipe
2025-11-28T18:22:32.058863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.058873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.058896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 61 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.058908Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.058922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.058929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.058940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.058946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.058950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.059699Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 61 partNo 0
2025-11-28T18:22:32.059737Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 61 partNo 0 FormedBlobsCount 0 NewHead: Offset 61 PartNo 0 PackedSize 262214 count 1 nextOffset 62 batches 1
2025-11-28T18:22:32.059743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.059745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.059748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.059840Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 61,1 HeadOffset 61 endOffset 61 curOffset 62 d0000000000_00000000000000000061_00000_0000000001_00000? size 262204 WTime 1764354152059
2025-11-28T18:22:32.059891Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.059907Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 61 partNo 0 count 1 size 262204
2025-11-28T18:22:32.061946Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 61 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.061965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.061979Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.061985Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 61 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.061987Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.061995Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 61, partNo: 0, Offset: 61 is stored on disk
2025-11-28T18:22:32.062021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.062028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.062029Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.062031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.062032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.062037Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.062044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.062110Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:32.062383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045039:3546] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 61
    Offset: 61
    WriteTimestampMS: 1764354152058
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.062986Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.063307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045043:3548], now have 1 active actors on pipe
2025-11-28T18:22:32.063424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.063432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.063460Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|e016e353-e0f1653f-86d76199-5d801bf5_62 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.063470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.063473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.063477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.063479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.063482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.063489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.063495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.063497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.063505Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.063514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.063559Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|e016e353-e0f1653f-86d76199-5d801bf5_62"
    Status: Active
    SeqNo: 0
  }
}
2025-11-28T18:22:32.063720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045043:3548] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|e016e353-e0f1653f-86d76199-5d801bf5_62"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 62
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.065527Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.065848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045046:3549], now have 1 active actors on pipe
2025-11-28T18:22:32.066180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.066195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.066215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 62 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.066225Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.066239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.066246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.066255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.066259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.066263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.067290Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 62 partNo 0
2025-11-28T18:22:32.067335Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 62 partNo 0 FormedBlobsCount 0 NewHead: Offset 62 PartNo 0 PackedSize 262214 count 1 nextOffset 63 batches 1
2025-11-28T18:22:32.067348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.067351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.067355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.067447Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 62,1 HeadOffset 62 endOffset 62 curOffset 63 d0000000000_00000000000000000062_00000_0000000001_00000? size 262204 WTime 1764354152067
2025-11-28T18:22:32.067498Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.067515Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 62 partNo 0 count 1 size 262204
2025-11-28T18:22:32.069255Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 62 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.069279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.069298Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.069308Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.069311Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.069320Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 62, partNo: 0, Offset: 62 is stored on disk
2025-11-28T18:22:32.069351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.069358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.069360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.069363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.069365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.069372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.069382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.069461Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:32.069694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045046:3549] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 62
    Offset: 62
    WriteTimestampMS: 1764354152065
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 1
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.070406Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.070752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045052:3553], now have 1 active actors on pipe
2025-11-28T18:22:32.070795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.070804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.070837Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|accb2f55-ab468241-7be6ef23-a03c10f7_63 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.070848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.070850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.070854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.070856Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.070859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.070870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.070872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.070875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.070884Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.070895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.070943Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:32.071122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045052:3553] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|accb2f55-ab468241-7be6ef23-a03c10f7_63"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|accb2f55-ab468241-7be6ef23-a03c10f7_63"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 63
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.072818Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.073193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045055:3554], now have 1 active actors on pipe
2025-11-28T18:22:32.073538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.073550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.073568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 63 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.073577Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.073587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.073594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.073605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.073611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.073617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.074440Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 63 partNo 0
2025-11-28T18:22:32.074489Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 63 partNo 0 FormedBlobsCount 0 NewHead: Offset 63 PartNo 0 PackedSize 262214 count 1 nextOffset 64 batches 1
2025-11-28T18:22:32.074497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.074500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.074502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.074593Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 63,1 HeadOffset 63 endOffset 63 curOffset 64 d0000000000_00000000000000000063_00000_0000000001_00000? size 262204 WTime 1764354152074
2025-11-28T18:22:32.074638Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.074659Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 63 partNo 0 count 1 size 262204
2025-11-28T18:22:32.076669Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 63 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.076697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.076733Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.076742Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 63 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.076746Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.076755Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 63, partNo: 0, Offset: 63 is stored on disk
2025-11-28T18:22:32.076787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.076796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.076799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.076801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.076803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.076812Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000032_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000033_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000034_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000035_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000036_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000037_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000038_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000039_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000040_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000041_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000042_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000043_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000044_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000045_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000046_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000047_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000048_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000049_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000050_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000051_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000052_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000053_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000054_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000055_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000056_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000057_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000058_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000059_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000060_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000061_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000062_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000063_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.076922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 32 keys were taken away. Let's read 8390528 bytes
2025-11-28T18:22:32.076936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.076960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 32 blobs
2025-11-28T18:22:32.076968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000032_00000_0000000001_00000?
2025-11-28T18:22:32.076978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000033_00000_0000000001_00000?
2025-11-28T18:22:32.076992Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000034_00000_0000000001_00000?
2025-11-28T18:22:32.077010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000035_00000_0000000001_00000?
2025-11-28T18:22:32.077022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000036_00000_0000000001_00000?
2025-11-28T18:22:32.077044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000037_00000_0000000001_00000?
2025-11-28T18:22:32.077014Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.077059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000038_00000_0000000001_00000?
2025-11-28T18:22:32.077072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000039_00000_0000000001_00000?
2025-11-28T18:22:32.077080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000040_00000_0000000001_00000?
2025-11-28T18:22:32.077112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000041_00000_0000000001_00000?
2025-11-28T18:22:32.077125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000042_00000_0000000001_00000?
2025-11-28T18:22:32.077139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000043_00000_0000000001_00000?
2025-11-28T18:22:32.077152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000044_00000_0000000001_00000?
2025-11-28T18:22:32.077165Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000045_00000_0000000001_00000?
2025-11-28T18:22:32.077178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000046_00000_0000000001_00000?
2025-11-28T18:22:32.077193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000047_00000_0000000001_00000?
2025-11-28T18:22:32.077205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000048_00000_0000000001_00000?
2025-11-28T18:22:32.077250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000049_00000_0000000001_00000?
2025-11-28T18:22:32.077264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000050_00000_0000000001_00000?
2025-11-28T18:22:32.077278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000051_00000_0000000001_00000?
2025-11-28T18:22:32.077292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000052_00000_0000000001_00000?
2025-11-28T18:22:32.077306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000053_00000_0000000001_00000?
CallPersQueueGRPC response:
2025-11-28T18:22:32.077320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000054_00000_0000000001_00000?
2025-11-28T18:22:32.077334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000055_00000_0000000001_00000?
2025-11-28T18:22:32.077347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000056_00000_0000000001_00000?
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 63
    Offset: 63
    WriteTimestampMS: 1764354152073
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:32.077361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000057_00000_0000000001_00000?
2025-11-28T18:22:32.077374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000058_00000_0000000001_00000?
2025-11-28T18:22:32.077389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000059_00000_0000000001_00000?
CallPersQueueGRPC request to localhost:2025-11-28T18:22:32.077403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000060_00000_0000000001_00000?
19639
2025-11-28T18:22:32.077417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000061_00000_0000000001_00000?
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.077431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000062_00000_0000000001_00000?
2025-11-28T18:22:32.077445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000063_00000_0000000001_00000?
2025-11-28T18:22:32.077460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 32 blobs for compaction
2025-11-28T18:22:32.077476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045055:3554] destroyed
2025-11-28T18:22:32.077492Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 32 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:32.077502Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 33 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:32.077506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 34 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:32.077515Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 35 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:32.077520Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 36 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:32.077529Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 37 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:32.077534Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 38 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:32.077542Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 39 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:32.077546Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 40 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:32.077550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 41 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:32.077559Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 42 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:32.077562Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 43 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:32.077567Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 44 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:32.077576Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 45 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:32.077580Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 46 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:32.077590Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 47 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:32.077593Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 48 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:32.077596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 49 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:32.077600Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 50 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:32.077604Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 51 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:32.077607Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 52 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:32.077611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 53 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:32.077616Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 54 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.077625Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 55 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.077628Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 56 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.077632Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 57 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.077640Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 58 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.077644Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 59 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.077647Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 60 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.077651Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 61 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.077660Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 62 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.077663Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 63 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.077679Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 32 blobs are from cache.
2025-11-28T18:22:32.077716Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 32 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-28T18:22:32.077728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/32] d0000000000_00000000000000000032_00000_0000000001_00000?
2025-11-28T18:22:32.077729Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 33 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.077733Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 34 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000032_00000_0000000001_00000?, parameters.CurOffset=32
2025-11-28T18:22:32.077736Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 35 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077740Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 36 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077743Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 37 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077746Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 38 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077751Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 39 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077754Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 40 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077757Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 41 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077761Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 42 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077764Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 43 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077769Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 44 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077772Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 45 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077776Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 46 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077786Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 47 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077790Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 48 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077795Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 49 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077803Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 50 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077806Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 51 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077809Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 52 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077812Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 53 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077816Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 54 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 32.0/1
2025-11-28T18:22:32.077825Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 55 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 32 partNo 0
2025-11-28T18:22:32.077830Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 56 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077834Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 57 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077838Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 58 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077841Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 59 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 32 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 262214 count 1 nextOffset 33 batches 1
2025-11-28T18:22:32.077852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 32:0 appended
2025-11-28T18:22:32.077852Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 60 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077856Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 61 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/32] d0000000000_00000000000000000033_00000_0000000001_00000?
2025-11-28T18:22:32.077859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.077859Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000033_00000_0000000001_00000?, parameters.CurOffset=33
2025-11-28T18:22:32.077862Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 63 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.077920Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.077937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 33.0/1
2025-11-28T18:22:32.077954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 33 partNo 0
2025-11-28T18:22:32.078288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 33 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 524387 count 2 nextOffset 34 batches 1
2025-11-28T18:22:32.078304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 33:0 appended
2025-11-28T18:22:32.078309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/32] d0000000000_00000000000000000034_00000_0000000001_00000?
2025-11-28T18:22:32.078311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.078314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000034_00000_0000000001_00000?, parameters.CurOffset=34
2025-11-28T18:22:32.078374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 34.0/1
2025-11-28T18:22:32.078387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 34 partNo 0
2025-11-28T18:22:32.078399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 34 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 786601 count 3 nextOffset 35 batches 2
2025-11-28T18:22:32.078408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 34:0 appended
2025-11-28T18:22:32.078412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/32] d0000000000_00000000000000000035_00000_0000000001_00000?
2025-11-28T18:22:32.078414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.078417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000035_00000_0000000001_00000?, parameters.CurOffset=35
2025-11-28T18:22:32.078492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 35.0/1
2025-11-28T18:22:32.078506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 35 partNo 0
2025-11-28T18:22:32.078650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 35 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 1048774 count 4 nextOffset 36 batches 2
2025-11-28T18:22:32.078664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 35:0 appended
2025-11-28T18:22:32.078669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/32] d0000000000_00000000000000000036_00000_0000000001_00000?
2025-11-28T18:22:32.078672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.078675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000036_00000_0000000001_00000?, parameters.CurOffset=36
2025-11-28T18:22:32.078730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 36.0/1
2025-11-28T18:22:32.078743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 36 partNo 0
2025-11-28T18:22:32.078759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 36 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 1310988 count 5 nextOffset 37 batches 3
2025-11-28T18:22:32.078767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 36:0 appended
2025-11-28T18:22:32.078771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/32] d0000000000_00000000000000000037_00000_0000000001_00000?
2025-11-28T18:22:32.078773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.078775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000037_00000_0000000001_00000?, parameters.CurOffset=37
2025-11-28T18:22:32.078829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 37.0/1
2025-11-28T18:22:32.078840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 37 partNo 0
2025-11-28T18:22:32.078967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 37 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 1573161 count 6 nextOffset 38 batches 3
2025-11-28T18:22:32.078980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 37:0 appended
2025-11-28T18:22:32.078986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/32] d0000000000_00000000000000000038_00000_0000000001_00000?
2025-11-28T18:22:32.078988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.078991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000038_00000_0000000001_00000?, parameters.CurOffset=38
2025-11-28T18:22:32.079058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 38.0/1
2025-11-28T18:22:32.079070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 38 partNo 0
2025-11-28T18:22:32.079079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 38 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 1835375 count 7 nextOffset 39 batches 4
2025-11-28T18:22:32.079087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 38:0 appended
2025-11-28T18:22:32.079091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/32] d0000000000_00000000000000000039_00000_0000000001_00000?
2025-11-28T18:22:32.079093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.079098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000039_00000_0000000001_00000?, parameters.CurOffset=39
2025-11-28T18:22:32.079169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 39.0/1
2025-11-28T18:22:32.079181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 39 partNo 0
2025-11-28T18:22:32.079310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 39 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 2097548 count 8 nextOffset 40 batches 4
2025-11-28T18:22:32.079322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 39:0 appended
2025-11-28T18:22:32.079326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[8/32] d0000000000_00000000000000000040_00000_0000000001_00000?
2025-11-28T18:22:32.079334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.079337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000040_00000_0000000001_00000?, parameters.CurOffset=40
2025-11-28T18:22:32.079390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 40.0/1
2025-11-28T18:22:32.079406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 40 partNo 0
2025-11-28T18:22:32.079434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 40 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 2359762 count 9 nextOffset 41 batches 5
2025-11-28T18:22:32.079437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 40:0 appended
2025-11-28T18:22:32.079440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[9/32] d0000000000_00000000000000000041_00000_0000000001_00000?
2025-11-28T18:22:32.079443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.079446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000041_00000_0000000001_00000?, parameters.CurOffset=41
2025-11-28T18:22:32.079511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 41.0/1
2025-11-28T18:22:32.079526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 41 partNo 0
2025-11-28T18:22:32.079643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 41 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 2621935 count 10 nextOffset 42 batches 5
2025-11-28T18:22:32.079657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 41:0 appended
2025-11-28T18:22:32.079662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[10/32] d0000000000_00000000000000000042_00000_0000000001_00000?
2025-11-28T18:22:32.079664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.079666Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000042_00000_0000000001_00000?, parameters.CurOffset=42
2025-11-28T18:22:32.079733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 42.0/1
2025-11-28T18:22:32.079744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 42 partNo 0
2025-11-28T18:22:32.079756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 42 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 2884149 count 11 nextOffset 43 batches 6
2025-11-28T18:22:32.079760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 42:0 appended
2025-11-28T18:22:32.079763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[11/32] d0000000000_00000000000000000043_00000_0000000001_00000?
2025-11-28T18:22:32.079765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.079767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000043_00000_0000000001_00000?, parameters.CurOffset=43
2025-11-28T18:22:32.079831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 43.0/1
2025-11-28T18:22:32.079841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 43 partNo 0
2025-11-28T18:22:32.079966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 43 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 3146322 count 12 nextOffset 44 batches 6
2025-11-28T18:22:32.079978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 43:0 appended
2025-11-28T18:22:32.079983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[12/32] d0000000000_00000000000000000044_00000_0000000001_00000?
2025-11-28T18:22:32.079985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.079987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000044_00000_0000000001_00000?, parameters.CurOffset=44
2025-11-28T18:22:32.080042Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 44.0/1
2025-11-28T18:22:32.080053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 44 partNo 0
2025-11-28T18:22:32.080063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 44 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 3408536 count 13 nextOffset 45 batches 7
2025-11-28T18:22:32.080070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 44:0 appended
2025-11-28T18:22:32.080074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[13/32] d0000000000_00000000000000000045_00000_0000000001_00000?
2025-11-28T18:22:32.080075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.080077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000045_00000_0000000001_00000?, parameters.CurOffset=45
2025-11-28T18:22:32.080140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 45.0/1
2025-11-28T18:22:32.080148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 45 partNo 0
2025-11-28T18:22:32.080280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 45 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 3670709 count 14 nextOffset 46 batches 7
2025-11-28T18:22:32.080291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 45:0 appended
2025-11-28T18:22:32.080295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[14/32] d0000000000_00000000000000000046_00000_0000000001_00000?
2025-11-28T18:22:32.080297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.080299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000046_00000_0000000001_00000?, parameters.CurOffset=46
2025-11-28T18:22:32.080364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 46.0/1
2025-11-28T18:22:32.080373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 46 partNo 0
2025-11-28T18:22:32.080383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 46 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 3932923 count 15 nextOffset 47 batches 8
2025-11-28T18:22:32.080390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 46:0 appended
2025-11-28T18:22:32.080392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[15/32] d0000000000_00000000000000000047_00000_0000000001_00000?
2025-11-28T18:22:32.080394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.080396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000047_00000_0000000001_00000?, parameters.CurOffset=47
2025-11-28T18:22:32.080462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 47.0/1
2025-11-28T18:22:32.080470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 47 partNo 0
2025-11-28T18:22:32.080576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 47 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 4195096 count 16 nextOffset 48 batches 8
2025-11-28T18:22:32.080587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 47:0 appended
2025-11-28T18:22:32.080592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[16/32] d0000000000_00000000000000000048_00000_0000000001_00000?
2025-11-28T18:22:32.080593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.080596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000048_00000_0000000001_00000?, parameters.CurOffset=48
2025-11-28T18:22:32.080663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 48.0/1
2025-11-28T18:22:32.080672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 48 partNo 0
2025-11-28T18:22:32.080682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 48 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 4457310 count 17 nextOffset 49 batches 9
2025-11-28T18:22:32.080684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 48:0 appended
2025-11-28T18:22:32.080687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[17/32] d0000000000_00000000000000000049_00000_0000000001_00000?
2025-11-28T18:22:32.080689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.080691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000049_00000_0000000001_00000?, parameters.CurOffset=49
2025-11-28T18:22:32.080752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 49.0/1
2025-11-28T18:22:32.080761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 49 partNo 0
2025-11-28T18:22:32.080865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 49 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 4719483 count 18 nextOffset 50 batches 9
2025-11-28T18:22:32.080874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 49:0 appended
2025-11-28T18:22:32.080878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[18/32] d0000000000_00000000000000000050_00000_0000000001_00000?
2025-11-28T18:22:32.080880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.080884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000050_00000_0000000001_00000?, parameters.CurOffset=50
2025-11-28T18:22:32.080934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 50.0/1
2025-11-28T18:22:32.080943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 50 partNo 0
2025-11-28T18:22:32.080953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 50 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 4981697 count 19 nextOffset 51 batches 10
2025-11-28T18:22:32.080957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 50:0 appended
2025-11-28T18:22:32.080960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[19/32] d0000000000_00000000000000000051_00000_0000000001_00000?
2025-11-28T18:22:32.080961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.080963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000051_00000_0000000001_00000?, parameters.CurOffset=51
2025-11-28T18:22:32.081022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 51.0/1
2025-11-28T18:22:32.081031Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 51 partNo 0
2025-11-28T18:22:32.081138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 51 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 5243870 count 20 nextOffset 52 batches 10
2025-11-28T18:22:32.081148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 51:0 appended
2025-11-28T18:22:32.081152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[20/32] d0000000000_00000000000000000052_00000_0000000001_00000?
2025-11-28T18:22:32.081155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.081157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000052_00000_0000000001_00000?, parameters.CurOffset=52
2025-11-28T18:22:32.081211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 52.0/1
2025-11-28T18:22:32.081220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 52 partNo 0
2025-11-28T18:22:32.081231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 52 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 5506084 count 21 nextOffset 53 batches 11
2025-11-28T18:22:32.081237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 52:0 appended
2025-11-28T18:22:32.081240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[21/32] d0000000000_00000000000000000053_00000_0000000001_00000?
2025-11-28T18:22:32.081242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.081244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000053_00000_0000000001_00000?, parameters.CurOffset=53
2025-11-28T18:22:32.081294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 53.0/1
2025-11-28T18:22:32.081303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 53 partNo 0
2025-11-28T18:22:32.081407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 53 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 5768257 count 22 nextOffset 54 batches 11
2025-11-28T18:22:32.081417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 53:0 appended
2025-11-28T18:22:32.081421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[22/32] d0000000000_00000000000000000054_00000_0000000001_00000?
2025-11-28T18:22:32.081423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.081426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000054_00000_0000000001_00000?, parameters.CurOffset=54
2025-11-28T18:22:32.081494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 54.0/1
2025-11-28T18:22:32.081508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 54 partNo 0
2025-11-28T18:22:32.081519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 54 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 6030471 count 23 nextOffset 55 batches 12
2025-11-28T18:22:32.081523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 54:0 appended
2025-11-28T18:22:32.081526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[23/32] d0000000000_00000000000000000055_00000_0000000001_00000?
2025-11-28T18:22:32.081528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.081533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000055_00000_0000000001_00000?, parameters.CurOffset=55
2025-11-28T18:22:32.081583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 55.0/1
2025-11-28T18:22:32.081587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 55 partNo 0
2025-11-28T18:22:32.082085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 55 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 6292644 count 24 nextOffset 56 batches 12
2025-11-28T18:22:32.082102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 55:0 appended
2025-11-28T18:22:32.082107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[24/32] d0000000000_00000000000000000056_00000_0000000001_00000?
2025-11-28T18:22:32.082108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.082111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000056_00000_0000000001_00000?, parameters.CurOffset=56
2025-11-28T18:22:32.082166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 56.0/1
2025-11-28T18:22:32.082177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 56 partNo 0
2025-11-28T18:22:32.082201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 56 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 6554858 count 25 nextOffset 57 batches 13
2025-11-28T18:22:32.082210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 56:0 appended
2025-11-28T18:22:32.082214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[25/32] d0000000000_00000000000000000057_00000_0000000001_00000?
2025-11-28T18:22:32.082216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.082218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000057_00000_0000000001_00000?, parameters.CurOffset=57
2025-11-28T18:22:32.082267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 57.0/1
2025-11-28T18:22:32.082277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 57 partNo 0
2025-11-28T18:22:32.082785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 57 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 6817031 count 26 nextOffset 58 batches 13
2025-11-28T18:22:32.082800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 57:0 appended
2025-11-28T18:22:32.082806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[26/32] d0000000000_00000000000000000058_00000_0000000001_00000?
2025-11-28T18:22:32.082808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.082810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000058_00000_0000000001_00000?, parameters.CurOffset=58
2025-11-28T18:22:32.082867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 58.0/1
2025-11-28T18:22:32.082877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 58 partNo 0
2025-11-28T18:22:32.082888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 58 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 7079245 count 27 nextOffset 59 batches 14
2025-11-28T18:22:32.082894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 58:0 appended
2025-11-28T18:22:32.082897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[27/32] d0000000000_00000000000000000059_00000_0000000001_00000?
2025-11-28T18:22:32.082899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.082902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000059_00000_0000000001_00000?, parameters.CurOffset=59
2025-11-28T18:22:32.082966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 59.0/1
2025-11-28T18:22:32.082976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 59 partNo 0
2025-11-28T18:22:32.083464Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 59 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 7341418 count 28 nextOffset 60 batches 14
2025-11-28T18:22:32.083479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 59:0 appended
2025-11-28T18:22:32.083483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[28/32] d0000000000_00000000000000000060_00000_0000000001_00000?
2025-11-28T18:22:32.083485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.083488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000060_00000_0000000001_00000?, parameters.CurOffset=60
2025-11-28T18:22:32.083542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 60.0/1
2025-11-28T18:22:32.083552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 60 partNo 0
2025-11-28T18:22:32.083565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 60 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 7603632 count 29 nextOffset 61 batches 15
2025-11-28T18:22:32.083568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 60:0 appended
2025-11-28T18:22:32.083571Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[29/32] d0000000000_00000000000000000061_00000_0000000001_00000?
2025-11-28T18:22:32.083573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.083575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000061_00000_0000000001_00000?, parameters.CurOffset=61
2025-11-28T18:22:32.083628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 61.0/1
2025-11-28T18:22:32.083638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 61 partNo 0
2025-11-28T18:22:32.084126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 61 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 7865805 count 30 nextOffset 62 batches 15
2025-11-28T18:22:32.084141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 61:0 appended
2025-11-28T18:22:32.084145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[30/32] d0000000000_00000000000000000062_00000_0000000001_00000?
2025-11-28T18:22:32.084147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.084150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000062_00000_0000000001_00000?, parameters.CurOffset=62
2025-11-28T18:22:32.084213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 62.0/1
2025-11-28T18:22:32.084224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 62 partNo 0
2025-11-28T18:22:32.091083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 62 partNo 0 result is x0000000000_00000000000000000031_00000_0000000031_00000 size 8128009
2025-11-28T18:22:32.091122Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1110: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000031_00000_0000000031_00000 new key d0000000000_00000000000000000031_00000_0000000031_00000 size 8128009 WTime 1764354152090
2025-11-28T18:22:32.091160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 62 partNo 0 FormedBlobsCount 1 NewHead: Offset 62 PartNo 0 PackedSize 262214 count 1 nextOffset 63 batches 1
2025-11-28T18:22:32.091169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 62:0 appended
2025-11-28T18:22:32.091182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[31/32] d0000000000_00000000000000000063_00000_0000000001_00000?
2025-11-28T18:22:32.091191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.091197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000063_00000_0000000001_00000?, parameters.CurOffset=63
2025-11-28T18:22:32.091300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 63.0/1
2025-11-28T18:22:32.091311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 63 partNo 0
2025-11-28T18:22:32.091850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 63 partNo 0 FormedBlobsCount 0 NewHead: Offset 62 PartNo 0 PackedSize 524387 count 2 nextOffset 64 batches 1
2025-11-28T18:22:32.091870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 63:0 appended
2025-11-28T18:22:32.091923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 62,2 HeadOffset 31 endOffset 32 curOffset 64 d0000000000_00000000000000000062_00000_0000000002_00000| size 524387 WTime 1764354152091
2025-11-28T18:22:32.092044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045059:3556], now have 1 active actors on pipe
2025-11-28T18:22:32.092076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.092087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.092104Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.092118Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 31 partNo 0 count 31 size 8128009
2025-11-28T18:22:32.092130Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 62 partNo 0 count 2 size 524387
2025-11-28T18:22:32.092164Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|472cc2c8-8ca38fec-cdd87986-fdbdc0f7_64 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.092178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.092184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.092190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.092194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.092198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.092210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.092217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.092220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.092236Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.092296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.092364Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|472cc2c8-8ca38fec-cdd87986-fdbdc0f7_64"
    Status: Active
    SeqNo: 0
  }
}
2025-11-28T18:22:32.092794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045059:3556] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|472cc2c8-8ca38fec-cdd87986-fdbdc0f7_64"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 64
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.094793Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.095228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045065:3559], now have 1 active actors on pipe
2025-11-28T18:22:32.095570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.095581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.095604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 64 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.095620Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.095634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.095641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.095651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.095653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.095657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.096654Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 64 partNo 0
2025-11-28T18:22:32.096690Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 64 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 262214 count 1 nextOffset 65 batches 1
2025-11-28T18:22:32.096698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.096700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.096702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.096790Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 64,1 HeadOffset 64 endOffset 64 curOffset 65 d0000000000_00000000000000000064_00000_0000000001_00000? size 262204 WTime 1764354152096
2025-11-28T18:22:32.096833Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.096844Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 64 partNo 0 count 1 size 262204
2025-11-28T18:22:32.102429Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 64 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.102454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.102480Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 64 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.102492Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.102503Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.102515Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 64, partNo: 0, Offset: 64 is stored on disk
2025-11-28T18:22:32.102555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.102565Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.102567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.102569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.102572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.102580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:178: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-11-28T18:22:32.102593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.102674Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.102912Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 31 count 31 size 8128009 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.102926Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 62 count 2 size 524387 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.102939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.102952Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 31 parts 0 suffix '0' size 8128009
2025-11-28T18:22:32.102957Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 2 parts 0 suffix '124' size 524387
2025-11-28T18:22:32.103089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.103092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.103097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.103100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.103102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.103152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.103165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045065:3559] destroyed
2025-11-28T18:22:32.103172Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.103178Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000032_00000_0000000001_00000?(+) to d0000000000_00000000000000000032_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103181Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000033_00000_0000000001_00000?(+) to d0000000000_00000000000000000033_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103184Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000034_00000_0000000001_00000?(+) to d0000000000_00000000000000000034_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103185Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000035_00000_0000000001_00000?(+) to d0000000000_00000000000000000035_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103187Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000036_00000_0000000001_00000?(+) to d0000000000_00000000000000000036_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103189Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000037_00000_0000000001_00000?(+) to d0000000000_00000000000000000037_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103191Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000038_00000_0000000001_00000?(+) to d0000000000_00000000000000000038_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103193Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000039_00000_0000000001_00000?(+) to d0000000000_00000000000000000039_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103196Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000040_00000_0000000001_00000?(+) to d0000000000_00000000000000000040_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103198Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000041_00000_0000000001_00000?(+) to d0000000000_00000000000000000041_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103200Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000042_00000_0000000001_00000?(+) to d0000000000_00000000000000000042_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103203Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000043_00000_0000000001_00000?(+) to d0000000000_00000000000000000043_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103204Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000044_00000_0000000001_00000?(+) to d0000000000_00000000000000000044_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103207Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000045_00000_0000000001_00000?(+) to d0000000000_00000000000000000045_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103209Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000046_00000_0000000001_00000?(+) to d0000000000_00000000000000000046_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103211Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000047_00000_0000000001_00000?(+) to d0000000000_00000000000000000047_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103214Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000048_00000_0000000001_00000?(+) to d0000000000_00000000000000000048_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103223Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000049_00000_0000000001_00000?(+) to d0000000000_00000000000000000049_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103226Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000050_00000_0000000001_00000?(+) to d0000000000_00000000000000000050_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103228Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000051_00000_0000000001_00000?(+) to d0000000000_00000000000000000051_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103230Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000052_00000_0000000001_00000?(+) to d0000000000_00000000000000000052_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103232Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000053_00000_0000000001_00000?(+) to d0000000000_00000000000000000053_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103237Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000054_00000_0000000001_00000?(+) to d0000000000_00000000000000000054_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103239Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000055_00000_0000000001_00000?(+) to d0000000000_00000000000000000055_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103241Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000056_00000_0000000001_00000?(+) to d0000000000_00000000000000000056_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103243Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000057_00000_0000000001_00000?(+) to d0000000000_00000000000000000057_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103245Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000058_00000_0000000001_00000?(+) to d0000000000_00000000000000000058_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103249Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000059_00000_0000000001_00000?(+) to d0000000000_00000000000000000059_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103251Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000060_00000_0000000001_00000?(+) to d0000000000_00000000000000000060_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103254Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000061_00000_0000000001_00000?(+) to d0000000000_00000000000000000061_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103256Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000062_00000_0000000001_00000?(+) to d0000000000_00000000000000000062_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103258Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000063_00000_0000000001_00000?(+) to d0000000000_00000000000000000063_00000_0000000001_00000?(+)
2025-11-28T18:22:32.103261Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000031_00000_0000000001_00000|(+) to d0000000000_00000000000000000031_00000_0000000001_00000|(+)
2025-11-28T18:22:32.104162Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 32 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104174Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 33 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104179Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 34 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104184Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 35 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104189Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 36 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104193Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 37 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104198Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 38 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104202Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 39 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104207Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 40 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104211Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 41 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104216Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 42 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104220Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 43 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104224Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 44 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104228Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 45 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104233Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 46 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104237Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 47 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104244Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 48 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104251Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 49 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104255Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 50 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104258Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 51 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104262Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 52 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104266Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 53 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104270Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 54 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104273Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 55 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104282Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 56 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104285Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 57 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104289Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 58 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104292Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 59 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104296Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 60 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 61 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 62 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 63 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104318Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 31 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.104333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.104349Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.104361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.104363Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 32 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.104367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.104368Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 33 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.104371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.104372Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 34 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104375Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 35 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.104379Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 36 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104382Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 37 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104385Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 38 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104389Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 39 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104393Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 40 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104396Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 41 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104399Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 42 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104402Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 43 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104406Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 44 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104409Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 45 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104412Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 46 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104416Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 47 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104420Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 48 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104423Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 49 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104426Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 50 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104428Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 51 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104431Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 52 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104434Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 53 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104438Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 54 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104441Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 55 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104444Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 56 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104447Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 57 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104450Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 58 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104453Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 59 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104457Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 60 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104460Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 61 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104465Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104468Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 63 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.104478Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 0 suffix '124' size 262204
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 64
    Offset: 64
    WriteTimestampMS: 1764354152095
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.107399Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.107876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045074:3565], now have 1 active actors on pipe
2025-11-28T18:22:32.108034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.108044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.108078Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|3a7609ce-1e99e97f-134fbc50-fb7846b5_65 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.108091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.108095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.108102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.108108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.108112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.108121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.108129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.108132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.108146Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.108165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.108249Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|3a7609ce-1e99e97f-134fbc50-fb7846b5_65"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
2025-11-28T18:22:32.108594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045074:3565] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|3a7609ce-1e99e97f-134fbc50-fb7846b5_65"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 65
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.110454Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.110831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045077:3566], now have 1 active actors on pipe
2025-11-28T18:22:32.111168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.111184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.111207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 65 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.111220Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.111236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.111247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.111257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.111260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.111263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.112129Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 65 partNo 0
2025-11-28T18:22:32.112175Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 65 partNo 0 FormedBlobsCount 0 NewHead: Offset 65 PartNo 0 PackedSize 262214 count 1 nextOffset 66 batches 1
2025-11-28T18:22:32.112184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.112187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.112189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.112282Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 65,1 HeadOffset 65 endOffset 65 curOffset 66 d0000000000_00000000000000000065_00000_0000000001_00000? size 262204 WTime 1764354152111
2025-11-28T18:22:32.112335Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.112352Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 65 partNo 0 count 1 size 262204
2025-11-28T18:22:32.114321Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 65 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.114344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.114360Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 65 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.114364Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.114373Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.114383Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 65, partNo: 0, Offset: 65 is stored on disk
2025-11-28T18:22:32.114413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.114422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.114424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.114426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.114428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.114437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.114450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.114544Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:32.114851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045077:3566] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 65
    Offset: 65
    WriteTimestampMS: 1764354152110
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.115866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045081:3568], now have 1 active actors on pipe
2025-11-28T18:22:32.115935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.115946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.115983Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|2f787455-5680e61f-63a298e6-1eb216b5_66 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.115996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.115999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.116004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.116006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.116009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.116018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.116025Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.116028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.116039Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.116053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.115501Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.116125Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|2f787455-5680e61f-63a298e6-1eb216b5_66"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
2025-11-28T18:22:32.116776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045081:3568] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|2f787455-5680e61f-63a298e6-1eb216b5_66"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 66
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.118262Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.118683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045085:3570], now have 1 active actors on pipe
2025-11-28T18:22:32.119180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.119195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.119216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 66 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.119225Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.119236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.119243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.119252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.119258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.119262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.120080Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 66 partNo 0
2025-11-28T18:22:32.120124Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 66 partNo 0 FormedBlobsCount 0 NewHead: Offset 66 PartNo 0 PackedSize 262214 count 1 nextOffset 67 batches 1
2025-11-28T18:22:32.120132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.120135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.120137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.120228Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 66,1 HeadOffset 66 endOffset 66 curOffset 67 d0000000000_00000000000000000066_00000_0000000001_00000? size 262204 WTime 1764354152120
2025-11-28T18:22:32.120279Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.120297Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 66 partNo 0 count 1 size 262204
2025-11-28T18:22:32.122442Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 66 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.122483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.122500Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 66 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.122502Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.122510Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.122518Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 66, partNo: 0, Offset: 66 is stored on disk
2025-11-28T18:22:32.122547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.122556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.122558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.122560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.122562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.122569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.122579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.122692Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 66
    Offset: 66
    WriteTimestampMS: 1764354152119
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:32.123069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045085:3570] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.125808Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.126245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045089:3572], now have 1 active actors on pipe
2025-11-28T18:22:32.126331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.126342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.126379Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9ceab801-7a22971c-7ef88f80-30438c1f_67 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.126392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.126396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.126400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.126402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.126406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.126414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.126419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.126422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.126435Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.126452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.126514Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:32.126816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045089:3572] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9ceab801-7a22971c-7ef88f80-30438c1f_67"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9ceab801-7a22971c-7ef88f80-30438c1f_67"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 67
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.128800Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.129215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045092:3573], now have 1 active actors on pipe
2025-11-28T18:22:32.129610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.129629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.129650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 67 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.129660Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.129673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.129678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.129688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.129690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.129693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.130574Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 67 partNo 0
2025-11-28T18:22:32.130622Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 67 partNo 0 FormedBlobsCount 0 NewHead: Offset 67 PartNo 0 PackedSize 262214 count 1 nextOffset 68 batches 1
2025-11-28T18:22:32.130635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.130637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.130640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.130731Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 67,1 HeadOffset 67 endOffset 67 curOffset 68 d0000000000_00000000000000000067_00000_0000000001_00000? size 262204 WTime 1764354152130
2025-11-28T18:22:32.130783Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.130801Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 67 partNo 0 count 1 size 262204
2025-11-28T18:22:32.133370Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 67 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.133396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.133415Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.133418Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 67 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.133428Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.133437Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 67, partNo: 0, Offset: 67 is stored on disk
2025-11-28T18:22:32.133468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.133478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.133480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.133483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.133486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.133494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.133504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.133578Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 67
    Offset: 67
    WriteTimestampMS: 1764354152129
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.133915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045092:3573] destroyed
2025-11-28T18:22:32.134570Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.135071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045096:3575], now have 1 active actors on pipe
2025-11-28T18:22:32.135168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.135179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.135215Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|752a5725-3607db1d-a68f4c86-763b8ee2_68 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.135230Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.135232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.135236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.135238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.135241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.135248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.135251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.135253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.135264Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.135279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.135342Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:32.135557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045096:3575] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|752a5725-3607db1d-a68f4c86-763b8ee2_68"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|752a5725-3607db1d-a68f4c86-763b8ee2_68"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 68
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.137386Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.137767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045099:3576], now have 1 active actors on pipe
2025-11-28T18:22:32.138171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.138186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.138206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 68 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.138216Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.138231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.138238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.138248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.138251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.138254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.139318Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 68 partNo 0
2025-11-28T18:22:32.139364Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 68 partNo 0 FormedBlobsCount 0 NewHead: Offset 68 PartNo 0 PackedSize 262214 count 1 nextOffset 69 batches 1
2025-11-28T18:22:32.139373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.139380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.139384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.139477Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 68,1 HeadOffset 68 endOffset 68 curOffset 69 d0000000000_00000000000000000068_00000_0000000001_00000? size 262204 WTime 1764354152138
2025-11-28T18:22:32.139529Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.139548Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 68 partNo 0 count 1 size 262204
2025-11-28T18:22:32.141901Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 68 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.141926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.141947Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.141956Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 68 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.141961Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.141970Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 68, partNo: 0, Offset: 68 is stored on disk
2025-11-28T18:22:32.142002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.142005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.142006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.142008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.142010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.142016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.142025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.142130Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 68
    Offset: 68
    WriteTimestampMS: 1764354152137
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:32.142421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045099:3576] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.143000Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.143384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045103:3578], now have 1 active actors on pipe
2025-11-28T18:22:32.143425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.143430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.143467Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b822ae24-e5c59622-504bfb09-459ec8af_69 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.143480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.143482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.143486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.143487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.143490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.143498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.143500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.143503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.143513Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.143527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.143572Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b822ae24-e5c59622-504bfb09-459ec8af_69"
    Status: Active
    SeqNo: 0
  }
}
2025-11-28T18:22:32.143776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045103:3578] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b822ae24-e5c59622-504bfb09-459ec8af_69"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 69
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.145185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.145196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.145198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.145200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.145202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.145643Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.146078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045106:3579], now have 1 active actors on pipe
2025-11-28T18:22:32.146522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.146537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.146557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 69 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.146569Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.146582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.146589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.146598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.146604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.146607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.147643Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 69 partNo 0
2025-11-28T18:22:32.147691Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 69 partNo 0 FormedBlobsCount 0 NewHead: Offset 69 PartNo 0 PackedSize 262214 count 1 nextOffset 70 batches 1
2025-11-28T18:22:32.147704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.147707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.147710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.147813Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 69,1 HeadOffset 69 endOffset 69 curOffset 70 d0000000000_00000000000000000069_00000_0000000001_00000? size 262204 WTime 1764354152147
2025-11-28T18:22:32.147863Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.147880Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 69 partNo 0 count 1 size 262204
2025-11-28T18:22:32.150115Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 69 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.150147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.150167Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 69 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.150171Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.150184Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.150195Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 69, partNo: 0, Offset: 69 is stored on disk
2025-11-28T18:22:32.150235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.150245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.150257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.150259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.150261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.150272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.150286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.150390Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 69
    Offset: 69
    WriteTimestampMS: 1764354152146
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:32.150760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045106:3579] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.151342Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.151725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045110:3581], now have 1 active actors on pipe
2025-11-28T18:22:32.151811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.151824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.151861Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|537fb98b-8d7ccbba-5ed07fb4-caa70534_70 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.151876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.151879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.151884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.151894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.151898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.151908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.151911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.151914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.151929Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.151946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.151994Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|537fb98b-8d7ccbba-5ed07fb4-caa70534_70"
    Status: Active
    SeqNo: 0
  }
}
2025-11-28T18:22:32.152217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045110:3581] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|537fb98b-8d7ccbba-5ed07fb4-caa70534_70"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 70
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.153812Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.154216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045113:3582], now have 1 active actors on pipe
2025-11-28T18:22:32.154753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.154775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.154796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 70 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.154810Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.154829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.154839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.154849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.154870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.154876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.155751Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 70 partNo 0
2025-11-28T18:22:32.155801Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 70 partNo 0 FormedBlobsCount 0 NewHead: Offset 70 PartNo 0 PackedSize 262214 count 1 nextOffset 71 batches 1
2025-11-28T18:22:32.155818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.155822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.155825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.155927Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 70,1 HeadOffset 70 endOffset 70 curOffset 71 d0000000000_00000000000000000070_00000_0000000001_00000? size 262204 WTime 1764354152155
2025-11-28T18:22:32.155987Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.156004Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 70 partNo 0 count 1 size 262204
2025-11-28T18:22:32.158358Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 70 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.158389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.158408Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.158408Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 70 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.158419Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.158431Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 70, partNo: 0, Offset: 70 is stored on disk
2025-11-28T18:22:32.158486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.158494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.158496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.158498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.158500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.158511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.158524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.158598Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.158891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045113:3582] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 70
    Offset: 70
    WriteTimestampMS: 1764354152154
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.159515Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.159863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045117:3584], now have 1 active actors on pipe
2025-11-28T18:22:32.159944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.159961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.159996Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|51dd5fd5-77e713c5-b706a831-b91de7fd_71 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.160009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.160012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.160017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.160024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.160028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.160036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.160039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.160041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.160053Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.160066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.160112Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.160314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045117:3584] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|51dd5fd5-77e713c5-b706a831-b91de7fd_71"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|51dd5fd5-77e713c5-b706a831-b91de7fd_71"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 71
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.162206Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.162543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045120:3585], now have 1 active actors on pipe
2025-11-28T18:22:32.162940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.162954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.162972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 71 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.162981Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.162991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.162996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.163004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.163005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.163008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.163785Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 71 partNo 0
2025-11-28T18:22:32.163826Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 71 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 262214 count 1 nextOffset 72 batches 1
2025-11-28T18:22:32.163834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.163837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.163840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.163933Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 71,1 HeadOffset 71 endOffset 71 curOffset 72 d0000000000_00000000000000000071_00000_0000000001_00000? size 262204 WTime 1764354152163
2025-11-28T18:22:32.163981Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.163998Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 71 partNo 0 count 1 size 262204
2025-11-28T18:22:32.166182Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 71 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.166209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.166230Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.166234Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 71 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.166239Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.166248Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 71, partNo: 0, Offset: 71 is stored on disk
2025-11-28T18:22:32.166280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.166287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.166289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.166291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.166293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.166302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.166314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.166385Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.166669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045120:3585] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 71
    Offset: 71
    WriteTimestampMS: 1764354152162
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.167304Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.167707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045124:3587], now have 1 active actors on pipe
2025-11-28T18:22:32.167777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.167786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.167815Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|dee3bd6f-8dfe903d-e7162408-35ec10ce_72 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.167827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.167831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.167836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.167838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.167841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.167847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.167850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.167852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.167862Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.167876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.167929Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:32.168118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045124:3587] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|dee3bd6f-8dfe903d-e7162408-35ec10ce_72"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|dee3bd6f-8dfe903d-e7162408-35ec10ce_72"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 72
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.169970Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.170320Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045127:3588], now have 1 active actors on pipe
2025-11-28T18:22:32.170684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.170698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.170718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 72 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.170738Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.170755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.170759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.170767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.170770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.170773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.171720Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 72 partNo 0
2025-11-28T18:22:32.171754Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 72 partNo 0 FormedBlobsCount 0 NewHead: Offset 72 PartNo 0 PackedSize 262214 count 1 nextOffset 73 batches 1
2025-11-28T18:22:32.171762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.171766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.171769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.171854Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 72,1 HeadOffset 72 endOffset 72 curOffset 73 d0000000000_00000000000000000072_00000_0000000001_00000? size 262204 WTime 1764354152171
2025-11-28T18:22:32.171901Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.171917Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 72 partNo 0 count 1 size 262204
2025-11-28T18:22:32.174436Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 72 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.174482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.174502Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.174515Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.174525Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 72, partNo: 0, Offset: 72 is stored on disk
2025-11-28T18:22:32.174560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.174567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.174569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.174571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.174574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.174585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.174597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.174651Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 72 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.174683Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 72
    Offset: 72
    WriteTimestampMS: 1764354152170
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.175203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045127:3588] destroyed
2025-11-28T18:22:32.176129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045131:3590], now have 1 active actors on pipe
2025-11-28T18:22:32.176233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.176243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.176274Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|8889a8c9-d26ddb7f-2ea45473-4716b916_73 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.176287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.176290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.176294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.176296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.176299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.176308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.176311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.176313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.176326Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.176338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
2025-11-28T18:22:32.176584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045131:3590] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|8889a8c9-d26ddb7f-2ea45473-4716b916_73"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|8889a8c9-d26ddb7f-2ea45473-4716b916_73"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 73
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.178620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045134:3591], now have 1 active actors on pipe
2025-11-28T18:22:32.178946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.178959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.178980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 73 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.178993Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.179006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.179013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.179023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.179027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.179031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.179759Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 73 partNo 0
2025-11-28T18:22:32.179795Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 73 partNo 0 FormedBlobsCount 0 NewHead: Offset 73 PartNo 0 PackedSize 262214 count 1 nextOffset 74 batches 1
2025-11-28T18:22:32.179807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.179814Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.179817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.179912Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 73,1 HeadOffset 73 endOffset 73 curOffset 74 d0000000000_00000000000000000073_00000_0000000001_00000? size 262204 WTime 1764354152179
2025-11-28T18:22:32.179963Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.179978Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 73 partNo 0 count 1 size 262204
2025-11-28T18:22:32.182402Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 73 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.182430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.182452Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.182453Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 73 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.175614Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.176398Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.178247Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.182527Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.182544Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 73, partNo: 0, Offset: 73 is stored on disk
2025-11-28T18:22:32.182583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.182591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.182593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.182596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.182599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.182610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.182628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.182732Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.183092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045134:3591] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 73
    Offset: 73
    WriteTimestampMS: 1764354152178
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.191873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045138:3593], now have 1 active actors on pipe
2025-11-28T18:22:32.191905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.191912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.191953Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|769dfb31-bf3deb87-691494ff-234f2070_74 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.191970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.191980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.191991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.191997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.192002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.192013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.192020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.192023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.192041Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.192063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.190919Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.195336Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|769dfb31-bf3deb87-691494ff-234f2070_74"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|769dfb31-bf3deb87-691494ff-234f2070_74"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 74
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.198656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045138:3593] destroyed
2025-11-28T18:22:32.199695Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.202655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045141:3594], now have 1 active actors on pipe
2025-11-28T18:22:32.203141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.203159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.203185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 74 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.203199Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.203217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.203221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.203237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.203239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.203244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.204118Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 74 partNo 0
2025-11-28T18:22:32.204168Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 74 partNo 0 FormedBlobsCount 0 NewHead: Offset 74 PartNo 0 PackedSize 262214 count 1 nextOffset 75 batches 1
2025-11-28T18:22:32.204183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.204187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.204191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.204274Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 74,1 HeadOffset 74 endOffset 74 curOffset 75 d0000000000_00000000000000000074_00000_0000000001_00000? size 262204 WTime 1764354152202
2025-11-28T18:22:32.204320Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.204333Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 74 partNo 0 count 1 size 262204
2025-11-28T18:22:32.212901Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 74 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.212934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.212958Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 74 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.212960Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.212973Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.212987Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 74, partNo: 0, Offset: 74 is stored on disk
2025-11-28T18:22:32.213034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.213037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.213039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.213045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.213047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.213060Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.213074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.213173Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:32.213618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045141:3594] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 74
    Offset: 74
    WriteTimestampMS: 1764354152202
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 6
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.214299Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.214776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045145:3596], now have 1 active actors on pipe
2025-11-28T18:22:32.214902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.214914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.214955Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|40751df8-5b7e7035-85b4390a-5bafae15_75 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.214971Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.214975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.214981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.214983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.214989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.215001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.215004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.215007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.215023Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.215054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.215260Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|40751df8-5b7e7035-85b4390a-5bafae15_75"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|40751df8-5b7e7035-85b4390a-5bafae15_75"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 75
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.217845Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.215562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045145:3596] destroyed
2025-11-28T18:22:32.218439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045148:3597], now have 1 active actors on pipe
2025-11-28T18:22:32.219085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.219108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.219133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 75 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.219149Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.219173Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.219183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.219197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.219204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.219210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.220202Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 75 partNo 0
2025-11-28T18:22:32.220268Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 75 partNo 0 FormedBlobsCount 0 NewHead: Offset 75 PartNo 0 PackedSize 262214 count 1 nextOffset 76 batches 1
2025-11-28T18:22:32.220281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.220291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.220295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.220410Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 75,1 HeadOffset 75 endOffset 75 curOffset 76 d0000000000_00000000000000000075_00000_0000000001_00000? size 262204 WTime 1764354152219
2025-11-28T18:22:32.220472Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.220494Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 75 partNo 0 count 1 size 262204
2025-11-28T18:22:32.223513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 75 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.223544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.223561Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 75 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.223568Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.223579Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.223591Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 75, partNo: 0, Offset: 75 is stored on disk
2025-11-28T18:22:32.223637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.223645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.223647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.223650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.223653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.223669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.223687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.223813Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.224179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045148:3597] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 75
    Offset: 75
    WriteTimestampMS: 1764354152218
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.225193Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.225740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045152:3599], now have 1 active actors on pipe
2025-11-28T18:22:32.225769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.225790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.225837Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|2f90aed5-662bfdf1-be08a928-713635dd_76 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.225854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.225863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.225870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.225872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.225876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.225889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.225891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.225893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.225910Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.225930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.226014Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.226341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045152:3599] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|2f90aed5-662bfdf1-be08a928-713635dd_76"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|2f90aed5-662bfdf1-be08a928-713635dd_76"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 76
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.228696Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.229298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045155:3600], now have 1 active actors on pipe
2025-11-28T18:22:32.229705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.229716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.229736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 76 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.229747Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.229758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.229765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.229774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.229778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.229782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.230426Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 76 partNo 0
2025-11-28T18:22:32.230474Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 76 partNo 0 FormedBlobsCount 0 NewHead: Offset 76 PartNo 0 PackedSize 262214 count 1 nextOffset 77 batches 1
2025-11-28T18:22:32.230487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.230490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.230493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.230575Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 76,1 HeadOffset 76 endOffset 76 curOffset 77 d0000000000_00000000000000000076_00000_0000000001_00000? size 262204 WTime 1764354152230
2025-11-28T18:22:32.230627Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.230642Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 76 partNo 0 count 1 size 262204
2025-11-28T18:22:32.232864Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 76 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.232892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.232911Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 76 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.232914Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.232923Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.232933Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 76, partNo: 0, Offset: 76 is stored on disk
2025-11-28T18:22:32.232980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.232988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.232992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.232994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.232997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.233006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.233019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.233160Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.233536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045155:3600] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 76
    Offset: 76
    WriteTimestampMS: 1764354152229
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.234402Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.234795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045161:3602], now have 1 active actors on pipe
2025-11-28T18:22:32.234916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.234928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.234967Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|32c9e12f-ae2d48a6-15ed18a8-c11a3b71_77 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.234978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.234981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.234987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.234989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.234993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.235003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.235006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.235009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.235025Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.235045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.235111Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|32c9e12f-ae2d48a6-15ed18a8-c11a3b71_77"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
2025-11-28T18:22:32.235402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045161:3602] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|32c9e12f-ae2d48a6-15ed18a8-c11a3b71_77"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 77
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.238168Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.238640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045164:3605], now have 1 active actors on pipe
2025-11-28T18:22:32.239061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.239074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.239098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 77 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.239116Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.239146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.239154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.239167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.239174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.239180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.239849Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 77 partNo 0
2025-11-28T18:22:32.239892Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 77 partNo 0 FormedBlobsCount 0 NewHead: Offset 77 PartNo 0 PackedSize 262214 count 1 nextOffset 78 batches 1
2025-11-28T18:22:32.239901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.239907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.239910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.239988Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 77,1 HeadOffset 77 endOffset 77 curOffset 78 d0000000000_00000000000000000077_00000_0000000001_00000? size 262204 WTime 1764354152239
2025-11-28T18:22:32.240030Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.240038Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 77 partNo 0 count 1 size 262204
2025-11-28T18:22:32.242235Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 77 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.242257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.242276Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 77 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.242279Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.242291Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.242303Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 77, partNo: 0, Offset: 77 is stored on disk
2025-11-28T18:22:32.242347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.242354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.242356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.242359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.242361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.242371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.242386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.242478Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.242821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045164:3605] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 77
    Offset: 77
    WriteTimestampMS: 1764354152238
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.244228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045168:3607], now have 1 active actors on pipe
2025-11-28T18:22:32.244312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.244324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.244357Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|1e1577c2-c6cd9508-af69b982-5ad33a39_78 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.244369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.244372Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.244377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.244379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.244382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.244391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.244393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.244399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.244413Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.244425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.244742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045168:3607] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|1e1577c2-c6cd9508-af69b982-5ad33a39_78"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|1e1577c2-c6cd9508-af69b982-5ad33a39_78"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 78
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.245524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.245529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.245531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.245533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.245536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.243816Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.244476Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.246864Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.247247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045171:3608], now have 1 active actors on pipe
2025-11-28T18:22:32.247629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.247643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.247666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 78 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.247679Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.247692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.247709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.247720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.247721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.247725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.248510Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 78 partNo 0
2025-11-28T18:22:32.248559Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 78 partNo 0 FormedBlobsCount 0 NewHead: Offset 78 PartNo 0 PackedSize 262214 count 1 nextOffset 79 batches 1
2025-11-28T18:22:32.248569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.248573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.248575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.248663Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 78,1 HeadOffset 78 endOffset 78 curOffset 79 d0000000000_00000000000000000078_00000_0000000001_00000? size 262204 WTime 1764354152248
2025-11-28T18:22:32.248716Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.248742Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 78 partNo 0 count 1 size 262204
2025-11-28T18:22:32.252931Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 78 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.252959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.252979Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.252992Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.252992Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 78 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.253000Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 78, partNo: 0, Offset: 78 is stored on disk
2025-11-28T18:22:32.253032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.253037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.253039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.253041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.253042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.253050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.253061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
2025-11-28T18:22:32.253571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045171:3608] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 78
    Offset: 78
    WriteTimestampMS: 1764354152247
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.253162Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.254317Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.254861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045175:3610], now have 1 active actors on pipe
2025-11-28T18:22:32.255114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.255124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.255163Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a4ce67dd-21c46b76-afde497c-2728e01e_79 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.255176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.255181Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.255188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.255190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.255194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.255205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.255208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.255211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.255228Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.255243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.255318Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a4ce67dd-21c46b76-afde497c-2728e01e_79"
    Status: Active
    SeqNo: 0
  }
}
2025-11-28T18:22:32.255636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045175:3610] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a4ce67dd-21c46b76-afde497c-2728e01e_79"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 79
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.257438Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.258394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045178:3611], now have 1 active actors on pipe
2025-11-28T18:22:32.258860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.258874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.258898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 79 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.258910Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.258926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.258930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.258942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.258945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.258950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.259994Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 79 partNo 0
2025-11-28T18:22:32.260033Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 79 partNo 0 FormedBlobsCount 0 NewHead: Offset 79 PartNo 0 PackedSize 262214 count 1 nextOffset 80 batches 1
2025-11-28T18:22:32.260041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.260044Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.260047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.260151Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 79,1 HeadOffset 79 endOffset 79 curOffset 80 d0000000000_00000000000000000079_00000_0000000001_00000? size 262204 WTime 1764354152259
2025-11-28T18:22:32.260208Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.260221Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 79 partNo 0 count 1 size 262204
2025-11-28T18:22:32.263097Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 79 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.263125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.263149Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.263161Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.263178Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 79, partNo: 0, Offset: 79 is stored on disk
2025-11-28T18:22:32.263186Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 79 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.263216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.263220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.263222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.263224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.263228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.263238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.263251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.263340Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.263723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045178:3611] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 79
    Offset: 79
    WriteTimestampMS: 1764354152258
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.264672Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.265060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045182:3613], now have 1 active actors on pipe
2025-11-28T18:22:32.265161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.265171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.265210Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|1d1a73d5-3802accc-b4826c5c-1385ddd2_80 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.265223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.265229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.265234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.265236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.265243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.265252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.265254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.265256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.265271Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.265286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.265333Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:32.265587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045182:3613] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|1d1a73d5-3802accc-b4826c5c-1385ddd2_80"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|1d1a73d5-3802accc-b4826c5c-1385ddd2_80"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 80
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.267844Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.268342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045185:3614], now have 1 active actors on pipe
2025-11-28T18:22:32.268690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.268702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.268735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 80 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.268747Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.268762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.268766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.268776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.268777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.268783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.269748Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 80 partNo 0
2025-11-28T18:22:32.269783Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 80 partNo 0 FormedBlobsCount 0 NewHead: Offset 80 PartNo 0 PackedSize 262214 count 1 nextOffset 81 batches 1
2025-11-28T18:22:32.269792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.269795Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.269798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.269887Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 80,1 HeadOffset 80 endOffset 80 curOffset 81 d0000000000_00000000000000000080_00000_0000000001_00000? size 262204 WTime 1764354152269
2025-11-28T18:22:32.269931Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.269941Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 80 partNo 0 count 1 size 262204
2025-11-28T18:22:32.272560Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 80 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.272580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.272602Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.272613Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.272627Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 80, partNo: 0, Offset: 80 is stored on disk
2025-11-28T18:22:32.272665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.272668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.272670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.272672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.272675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.272684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.272696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.272817Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 80 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.272883Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.273508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045185:3614] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 80
    Offset: 80
    WriteTimestampMS: 1764354152268
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.274454Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.274962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045189:3616], now have 1 active actors on pipe
2025-11-28T18:22:32.275021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.275029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.275064Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|5401044a-dae9044d-28fe600e-395fa6ab_81 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.275075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.275079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.275085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.275087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.275090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.275100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.275102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.275105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.275119Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.275132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.275196Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|5401044a-dae9044d-28fe600e-395fa6ab_81"
    Status: Active
    SeqNo: 0
  }
}
2025-11-28T18:22:32.275467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045189:3616] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|5401044a-dae9044d-28fe600e-395fa6ab_81"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 81
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.277362Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.277899Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045192:3617], now have 1 active actors on pipe
2025-11-28T18:22:32.278374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.278384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.278406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 81 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.278416Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.278430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.278434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.278443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.278445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.278448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.279454Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 81 partNo 0
2025-11-28T18:22:32.279495Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 81 partNo 0 FormedBlobsCount 0 NewHead: Offset 81 PartNo 0 PackedSize 262214 count 1 nextOffset 82 batches 1
2025-11-28T18:22:32.279506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.279508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.279511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.279613Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 81,1 HeadOffset 81 endOffset 81 curOffset 82 d0000000000_00000000000000000081_00000_0000000001_00000? size 262204 WTime 1764354152279
2025-11-28T18:22:32.279659Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.279672Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 81 partNo 0 count 1 size 262204
2025-11-28T18:22:32.282273Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 81 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.282304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.282326Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.282335Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 81 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.282336Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.282346Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 81, partNo: 0, Offset: 81 is stored on disk
2025-11-28T18:22:32.282376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.282379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.282381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.282384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.282386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.282395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.282406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.282513Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 81
    Offset: 81
    WriteTimestampMS: 1764354152278
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
2025-11-28T18:22:32.283081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045192:3617] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.283787Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.284237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045196:3619], now have 1 active actors on pipe
2025-11-28T18:22:32.284370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.284380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.284418Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|c5851323-67b7f230-c28d7eb7-a25be1b4_82 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.284427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.284430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.284435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.284437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.284441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.284450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.284455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.284458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.284471Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.284483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.284613Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.284909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045196:3619] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|c5851323-67b7f230-c28d7eb7-a25be1b4_82"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|c5851323-67b7f230-c28d7eb7-a25be1b4_82"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 82
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.287181Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.287640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045200:3621], now have 1 active actors on pipe
2025-11-28T18:22:32.287996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.288006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.288083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 82 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.288128Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.288143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.288147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.288157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.288159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.288163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.289174Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 82 partNo 0
2025-11-28T18:22:32.289210Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 82 partNo 0 FormedBlobsCount 0 NewHead: Offset 82 PartNo 0 PackedSize 262214 count 1 nextOffset 83 batches 1
2025-11-28T18:22:32.289218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.289221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.289224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.289313Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 82,1 HeadOffset 82 endOffset 82 curOffset 83 d0000000000_00000000000000000082_00000_0000000001_00000? size 262204 WTime 1764354152289
2025-11-28T18:22:32.289355Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.289367Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 82 partNo 0 count 1 size 262204
2025-11-28T18:22:32.291642Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 82 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.291659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.291680Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.291680Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 82 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.291688Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.291696Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 82, partNo: 0, Offset: 82 is stored on disk
2025-11-28T18:22:32.291784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.291790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.291793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.291795Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.291797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.291804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.291813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.291900Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.292261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045200:3621] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 82
    Offset: 82
    WriteTimestampMS: 1764354152287
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.293194Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.293698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045204:3623], now have 1 active actors on pipe
2025-11-28T18:22:32.293714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.293720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.293754Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ccf35474-a4984450-f3cfcd6a-e5dd42a2_83 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.293762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.293765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.293770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.293772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.293775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.293784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.293786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.293789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.293801Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.293814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.293875Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ccf35474-a4984450-f3cfcd6a-e5dd42a2_83"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ccf35474-a4984450-f3cfcd6a-e5dd42a2_83"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 83
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.294183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045204:3623] destroyed
2025-11-28T18:22:32.296559Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.296969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045207:3624], now have 1 active actors on pipe
2025-11-28T18:22:32.297373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.297383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.297408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 83 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.297419Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.297433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.297436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.297447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.297449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.297452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.298758Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 83 partNo 0
2025-11-28T18:22:32.298807Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 83 partNo 0 FormedBlobsCount 0 NewHead: Offset 83 PartNo 0 PackedSize 262214 count 1 nextOffset 84 batches 1
2025-11-28T18:22:32.298817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.298820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.298826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.298932Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 83,1 HeadOffset 83 endOffset 83 curOffset 84 d0000000000_00000000000000000083_00000_0000000001_00000? size 262204 WTime 1764354152298
2025-11-28T18:22:32.298976Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.298990Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 83 partNo 0 count 1 size 262204
2025-11-28T18:22:32.301341Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 83 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.301362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.301381Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 83 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.301383Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.301394Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.301403Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 83, partNo: 0, Offset: 83 is stored on disk
2025-11-28T18:22:32.301434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.301437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.301439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.301441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.301443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.301451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.301461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.301546Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.301901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045207:3624] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 83
    Offset: 83
    WriteTimestampMS: 1764354152297
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.302657Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.303021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045211:3626], now have 1 active actors on pipe
2025-11-28T18:22:32.303132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.303140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.303171Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|71634f4c-ec4b2834-74e5ffb2-b1630e62_84 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.303178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.303181Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.303186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.303187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.303190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.303199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.303201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.303204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.303214Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.303224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.303280Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|71634f4c-ec4b2834-74e5ffb2-b1630e62_84"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|71634f4c-ec4b2834-74e5ffb2-b1630e62_84"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 84
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.305693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045211:3626] destroyed
2025-11-28T18:22:32.305706Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.306135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045214:3627], now have 1 active actors on pipe
2025-11-28T18:22:32.306604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.306616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.306638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 84 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.306649Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.306662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.306665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.306676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.306677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.306681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.308326Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 84 partNo 0
2025-11-28T18:22:32.308360Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 84 partNo 0 FormedBlobsCount 0 NewHead: Offset 84 PartNo 0 PackedSize 262214 count 1 nextOffset 85 batches 1
2025-11-28T18:22:32.308369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.308371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.308376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.308485Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 84,1 HeadOffset 84 endOffset 84 curOffset 85 d0000000000_00000000000000000084_00000_0000000001_00000? size 262204 WTime 1764354152308
2025-11-28T18:22:32.308539Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.308552Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 84 partNo 0 count 1 size 262204
2025-11-28T18:22:32.310930Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 84 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.310953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.310974Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.310981Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 84 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.310984Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.310994Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 84, partNo: 0, Offset: 84 is stored on disk
2025-11-28T18:22:32.311030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.311033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.311035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.311041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.311043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.311051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.311062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.311146Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 84
    Offset: 84
    WriteTimestampMS: 1764354152306
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.312122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045214:3627] destroyed
2025-11-28T18:22:32.312280Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.312683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045218:3629], now have 1 active actors on pipe
2025-11-28T18:22:32.312829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.312838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.312870Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|8d17fef0-c0af8d69-1937ec01-a879d040_85 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.312880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.312882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.312888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.312890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.312893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.312904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.312907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.312909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.312922Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.312932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.312991Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.313222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045218:3629] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|8d17fef0-c0af8d69-1937ec01-a879d040_85"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|8d17fef0-c0af8d69-1937ec01-a879d040_85"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 85
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.315497Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.316093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045221:3630], now have 1 active actors on pipe
2025-11-28T18:22:32.316253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.316261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.316284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 85 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.316294Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.316308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.316311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.316321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.316323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.316327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.317524Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 85 partNo 0
2025-11-28T18:22:32.317612Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 85 partNo 0 FormedBlobsCount 0 NewHead: Offset 85 PartNo 0 PackedSize 262214 count 1 nextOffset 86 batches 1
2025-11-28T18:22:32.317621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.317624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.317627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.317725Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 85,1 HeadOffset 85 endOffset 85 curOffset 86 d0000000000_00000000000000000085_00000_0000000001_00000? size 262204 WTime 1764354152317
2025-11-28T18:22:32.317766Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.317778Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 85 partNo 0 count 1 size 262204
2025-11-28T18:22:32.320216Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 85 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.320237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.320255Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.320259Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 85 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.320263Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.320271Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 85, partNo: 0, Offset: 85 is stored on disk
2025-11-28T18:22:32.320296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.320300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.320302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.320303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.320305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.320312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.320321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.320411Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.320699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045221:3630] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 85
    Offset: 85
    WriteTimestampMS: 1764354152316
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.321374Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.321716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045225:3632], now have 1 active actors on pipe
2025-11-28T18:22:32.321841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.321848Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.321876Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f84cd5dc-febb55e1-6edef4ac-edfddfbf_86 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.321883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.321886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.321890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.321892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.321894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.321903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.321905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.321907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.321917Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.321927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.322019Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:32.322249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045225:3632] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f84cd5dc-febb55e1-6edef4ac-edfddfbf_86"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f84cd5dc-febb55e1-6edef4ac-edfddfbf_86"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 86
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.324016Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.324374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045228:3633], now have 1 active actors on pipe
2025-11-28T18:22:32.324741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.324752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.324772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 86 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.324784Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.324794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.324801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.324809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.324815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.324827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.325670Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 86 partNo 0
2025-11-28T18:22:32.325709Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 86 partNo 0 FormedBlobsCount 0 NewHead: Offset 86 PartNo 0 PackedSize 262214 count 1 nextOffset 87 batches 1
2025-11-28T18:22:32.325720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.325724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.325727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.325807Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 86,1 HeadOffset 86 endOffset 86 curOffset 87 d0000000000_00000000000000000086_00000_0000000001_00000? size 262204 WTime 1764354152325
2025-11-28T18:22:32.325852Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.325866Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 86 partNo 0 count 1 size 262204
2025-11-28T18:22:32.327966Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 86 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.327987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.328003Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.328011Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 86 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.328017Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.328024Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 86, partNo: 0, Offset: 86 is stored on disk
2025-11-28T18:22:32.328049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.328055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.328057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.328059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.328061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.328069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.328077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.328158Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 86
    Offset: 86
    WriteTimestampMS: 1764354152324
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.328494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045228:3633] destroyed
2025-11-28T18:22:32.329014Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.329399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045232:3635], now have 1 active actors on pipe
2025-11-28T18:22:32.329494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.329504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.329533Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|5fdf8eab-6a70d603-b7673233-17ee4926_87 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.329544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.329547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.329551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.329552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.329555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.329562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.329568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.329570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.329579Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.329588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.329634Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|5fdf8eab-6a70d603-b7673233-17ee4926_87"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|5fdf8eab-6a70d603-b7673233-17ee4926_87"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 87
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.330421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045232:3635] destroyed
2025-11-28T18:22:32.331355Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.331666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045235:3636], now have 1 active actors on pipe
2025-11-28T18:22:32.331974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.331985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.332002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 87 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.332022Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.332035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.332042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.332050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.332055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.332058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.332787Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 87 partNo 0
2025-11-28T18:22:32.332819Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 87 partNo 0 FormedBlobsCount 0 NewHead: Offset 87 PartNo 0 PackedSize 262214 count 1 nextOffset 88 batches 1
2025-11-28T18:22:32.332825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.332831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.332833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.332891Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 87,1 HeadOffset 87 endOffset 87 curOffset 88 d0000000000_00000000000000000087_00000_0000000001_00000? size 262204 WTime 1764354152332
2025-11-28T18:22:32.332925Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.332937Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 87 partNo 0 count 1 size 262204
2025-11-28T18:22:32.334806Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 87 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.334830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.334849Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.334856Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.334869Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 87, partNo: 0, Offset: 87 is stored on disk
2025-11-28T18:22:32.334879Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 87 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.334900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.334903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.334904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.334906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.334908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.334915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.334923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.335011Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 87
    Offset: 87
    WriteTimestampMS: 1764354152332
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.335531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045235:3636] destroyed
2025-11-28T18:22:32.335881Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.336239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045239:3638], now have 1 active actors on pipe
2025-11-28T18:22:32.336326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.336334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.336361Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|1052e288-c990abe6-bb95f889-45010451_88 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.336373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.336376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.336381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.336383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.336386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.336393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.336398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.336400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.336408Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.336418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.336473Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.336662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045239:3638] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|1052e288-c990abe6-bb95f889-45010451_88"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|1052e288-c990abe6-bb95f889-45010451_88"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 88
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.338784Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.339116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045242:3639], now have 1 active actors on pipe
2025-11-28T18:22:32.339460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.339470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.339490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 88 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.339503Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.339516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.339523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.339531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.339537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.339540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.340289Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 88 partNo 0
2025-11-28T18:22:32.340319Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 88 partNo 0 FormedBlobsCount 0 NewHead: Offset 88 PartNo 0 PackedSize 262214 count 1 nextOffset 89 batches 1
2025-11-28T18:22:32.340325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.340327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.340329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.340389Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 88,1 HeadOffset 88 endOffset 88 curOffset 89 d0000000000_00000000000000000088_00000_0000000001_00000? size 262204 WTime 1764354152340
2025-11-28T18:22:32.340425Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.340436Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 88 partNo 0 count 1 size 262204
2025-11-28T18:22:32.342350Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 88 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.342367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.342381Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.342394Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.342397Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 88 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.342406Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 88, partNo: 0, Offset: 88 is stored on disk
2025-11-28T18:22:32.342437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.342450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.342451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.342453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.342455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.342478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.342492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.342566Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.342860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045242:3639] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 88
    Offset: 88
    WriteTimestampMS: 1764354152339
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.343496Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.343818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045246:3641], now have 1 active actors on pipe
2025-11-28T18:22:32.343954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.343967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.343998Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|427cc2b2-9412e11b-83e87f76-32e3a8d5_89 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.344010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.344013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.344017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.344020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.344023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.344031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.344037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.344040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.344051Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.344066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.344119Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.344301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045246:3641] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|427cc2b2-9412e11b-83e87f76-32e3a8d5_89"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|427cc2b2-9412e11b-83e87f76-32e3a8d5_89"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 89
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.345884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.345898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.345901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.345903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.345905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.346267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045249:3642], now have 1 active actors on pipe
2025-11-28T18:22:32.345939Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.346566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.346576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.346593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 89 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.346603Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.346611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.346616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.346623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.346624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.346626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.347194Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 89 partNo 0
2025-11-28T18:22:32.347237Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 89 partNo 0 FormedBlobsCount 0 NewHead: Offset 89 PartNo 0 PackedSize 262214 count 1 nextOffset 90 batches 1
2025-11-28T18:22:32.347247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.347250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.347254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.347333Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 89,1 HeadOffset 89 endOffset 89 curOffset 90 d0000000000_00000000000000000089_00000_0000000001_00000? size 262204 WTime 1764354152346
2025-11-28T18:22:32.347372Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.347380Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 89 partNo 0 count 1 size 262204
2025-11-28T18:22:32.349244Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 89 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.349266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.349285Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 89 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.349285Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.349294Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.349302Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 89, partNo: 0, Offset: 89 is stored on disk
2025-11-28T18:22:32.349334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.349340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.349342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.349344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.349346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.349352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.349363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.349429Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.349756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045249:3642] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 89
    Offset: 89
    WriteTimestampMS: 1764354152346
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.350422Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.350864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045253:3644], now have 1 active actors on pipe
2025-11-28T18:22:32.350930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.350940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.350974Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|7f75c5fe-fb6683f4-bfca7235-4643043d_90 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.350987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.350990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.350994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.350996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.350999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.351006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.351008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.351011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.351023Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.351034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.351114Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.351344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045253:3644] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|7f75c5fe-fb6683f4-bfca7235-4643043d_90"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|7f75c5fe-fb6683f4-bfca7235-4643043d_90"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 90
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.352982Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.353537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045256:3645], now have 1 active actors on pipe
2025-11-28T18:22:32.353688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.353699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.353724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 90 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.353737Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.353754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.353760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.353771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.353776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.353780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.354760Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 90 partNo 0
2025-11-28T18:22:32.354810Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 90 partNo 0 FormedBlobsCount 0 NewHead: Offset 90 PartNo 0 PackedSize 262214 count 1 nextOffset 91 batches 1
2025-11-28T18:22:32.354824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.354836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.354839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.354936Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 90,1 HeadOffset 90 endOffset 90 curOffset 91 d0000000000_00000000000000000090_00000_0000000001_00000? size 262204 WTime 1764354152354
2025-11-28T18:22:32.354984Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.354999Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 90 partNo 0 count 1 size 262204
2025-11-28T18:22:32.356875Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 90 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.356898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.356919Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.356925Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 90 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.356931Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.356940Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 90, partNo: 0, Offset: 90 is stored on disk
2025-11-28T18:22:32.356972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.356981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.356982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.356984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.356988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.356995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.357005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.357064Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:32.357350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045256:3645] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 90
    Offset: 90
    WriteTimestampMS: 1764354152353
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.357905Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.358261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045260:3647], now have 1 active actors on pipe
2025-11-28T18:22:32.358324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.358331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.358354Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|4e502201-6c0bf3bd-33d1b776-bf480bbe_91 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.358363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.358365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.358368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.358369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.358371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.358377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.358378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.358381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.358389Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.358399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.358441Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.358634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045260:3647] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|4e502201-6c0bf3bd-33d1b776-bf480bbe_91"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|4e502201-6c0bf3bd-33d1b776-bf480bbe_91"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 91
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.360680Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.360974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045263:3648], now have 1 active actors on pipe
2025-11-28T18:22:32.361353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.361367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.361389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 91 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.361402Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.361415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.361421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.361431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.361435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.361438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.362048Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 91 partNo 0
2025-11-28T18:22:32.362094Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 91 partNo 0 FormedBlobsCount 0 NewHead: Offset 91 PartNo 0 PackedSize 262214 count 1 nextOffset 92 batches 1
2025-11-28T18:22:32.362106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.362108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.362112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.362193Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 91,1 HeadOffset 91 endOffset 91 curOffset 92 d0000000000_00000000000000000091_00000_0000000001_00000? size 262204 WTime 1764354152361
2025-11-28T18:22:32.362239Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.362254Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 91 partNo 0 count 1 size 262204
2025-11-28T18:22:32.364230Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 91 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.364255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.364271Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 91 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.364276Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.364284Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.364292Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 91, partNo: 0, Offset: 91 is stored on disk
2025-11-28T18:22:32.364319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.364322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.364323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.364326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.364328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.364335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.364346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.364412Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.364683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045263:3648] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 91
    Offset: 91
    WriteTimestampMS: 1764354152361
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.365354Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.365695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045267:3650], now have 1 active actors on pipe
2025-11-28T18:22:32.365789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.365801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.365834Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f00b5baa-d4e2ff61-8d9d691b-8995d7a5_92 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.365844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.365846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.365850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.365851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.365853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.365859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.365861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.365862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.365870Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.365878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.365946Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.366143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045267:3650] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f00b5baa-d4e2ff61-8d9d691b-8995d7a5_92"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f00b5baa-d4e2ff61-8d9d691b-8995d7a5_92"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 92
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.368220Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.368553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045270:3651], now have 1 active actors on pipe
2025-11-28T18:22:32.368900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.368911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.368928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 92 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.368940Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.368950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.368952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.368959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.368960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.368962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.369472Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 92 partNo 0
2025-11-28T18:22:32.369506Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 92 partNo 0 FormedBlobsCount 0 NewHead: Offset 92 PartNo 0 PackedSize 262214 count 1 nextOffset 93 batches 1
2025-11-28T18:22:32.369518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.369521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.369523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.369609Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 92,1 HeadOffset 92 endOffset 92 curOffset 93 d0000000000_00000000000000000092_00000_0000000001_00000? size 262204 WTime 1764354152369
2025-11-28T18:22:32.369654Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.369670Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 92 partNo 0 count 1 size 262204
2025-11-28T18:22:32.373424Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 92 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.373449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.373471Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.373476Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 92 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.373481Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.373491Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 92, partNo: 0, Offset: 92 is stored on disk
2025-11-28T18:22:32.373532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.373540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.373541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.373544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.373545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.373554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.373565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.373642Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:32.373919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045270:3651] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 92
    Offset: 92
    WriteTimestampMS: 1764354152368
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.374658Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.375042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045274:3653], now have 1 active actors on pipe
2025-11-28T18:22:32.375169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.375180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.375213Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|fd86ebf0-84a8c152-fb759796-d9e6271b_93 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.375225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.375228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.375232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.375234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.375237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.375247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.375255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.375257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.375268Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.375283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.375336Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:32.375507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045274:3653] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|fd86ebf0-84a8c152-fb759796-d9e6271b_93"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|fd86ebf0-84a8c152-fb759796-d9e6271b_93"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 93
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.377332Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.377810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045277:3654], now have 1 active actors on pipe
2025-11-28T18:22:32.378283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.378298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.378319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 93 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.378335Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.378350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.378356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.378365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.378367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.378370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.379273Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 93 partNo 0
2025-11-28T18:22:32.379317Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 93 partNo 0 FormedBlobsCount 0 NewHead: Offset 93 PartNo 0 PackedSize 262214 count 1 nextOffset 94 batches 1
2025-11-28T18:22:32.379329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.379333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.379340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.379425Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 93,1 HeadOffset 93 endOffset 93 curOffset 94 d0000000000_00000000000000000093_00000_0000000001_00000? size 262204 WTime 1764354152379
2025-11-28T18:22:32.379475Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.379491Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 93 partNo 0 count 1 size 262204
2025-11-28T18:22:32.381375Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 93 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.381397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.381415Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 93 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.381416Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.381424Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.381432Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 93, partNo: 0, Offset: 93 is stored on disk
2025-11-28T18:22:32.381463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.381470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.381472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.381474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.381476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.381483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.381497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.381599Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-28T18:22:32.381979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045277:3654] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 93
    Offset: 93
    WriteTimestampMS: 1764354152378
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.382658Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.383075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045281:3656], now have 1 active actors on pipe
2025-11-28T18:22:32.386138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.386155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.386195Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|50e0917b-414d7183-6765c893-7030a60_94 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.386209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.386216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.386221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.386227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.386231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.386250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.386256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.386259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.386278Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.386296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.386357Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.386694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045281:3656] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|50e0917b-414d7183-6765c893-7030a60_94"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|50e0917b-414d7183-6765c893-7030a60_94"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 94
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.389015Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.389434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045284:3657], now have 1 active actors on pipe
2025-11-28T18:22:32.389835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.389850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.389886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 94 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.389904Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.389921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.389928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.389938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.389945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.389949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.391047Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 94 partNo 0
2025-11-28T18:22:32.391108Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 94 partNo 0 FormedBlobsCount 0 NewHead: Offset 94 PartNo 0 PackedSize 262214 count 1 nextOffset 95 batches 1
2025-11-28T18:22:32.391123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.391125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.391128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.391225Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 94,1 HeadOffset 94 endOffset 94 curOffset 95 d0000000000_00000000000000000094_00000_0000000001_00000? size 262204 WTime 1764354152391
2025-11-28T18:22:32.391278Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.391296Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 94 partNo 0 count 1 size 262204
2025-11-28T18:22:32.393124Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 94 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.393146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.393163Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 94 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.393166Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.393176Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.393189Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 94, partNo: 0, Offset: 94 is stored on disk
2025-11-28T18:22:32.393223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.393232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.393233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.393235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.393237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.393244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.393255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.393336Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 94
    Offset: 94
    WriteTimestampMS: 1764354152389
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:32.393721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045284:3657] destroyed
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-28T18:22:32.394322Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.394783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045288:3659], now have 1 active actors on pipe
2025-11-28T18:22:32.394878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.394888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.394922Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|926c7e3c-f3cc4044-fe506150-21702c09_95 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-28T18:22:32.394934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.394941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.394945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.394946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.394949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.394958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.394964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.394966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.394977Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-28T18:22:32.394992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.395061Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.395258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045288:3659] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|926c7e3c-f3cc4044-fe506150-21702c09_95"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|926c7e3c-f3cc4044-fe506150-21702c09_95"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 95
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.397305Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.397684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045291:3660], now have 1 active actors on pipe
2025-11-28T18:22:32.398004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.398019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.398042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 95 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-28T18:22:32.398057Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:640: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-28T18:22:32.398075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.398083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-28T18:22:32.398093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-28T18:22:32.398099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.398103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-28T18:22:32.399198Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1346: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 95 partNo 0
2025-11-28T18:22:32.399334Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1450: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 95 partNo 0 FormedBlobsCount 0 NewHead: Offset 95 PartNo 0 PackedSize 262214 count 1 nextOffset 96 batches 1
2025-11-28T18:22:32.399351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-28T18:22:32.399353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-28T18:22:32.399357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.399544Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1702: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 95,1 HeadOffset 95 endOffset 95 curOffset 96 d0000000000_00000000000000000095_00000_0000000001_00000? size 262204 WTime 1764354152399
2025-11-28T18:22:32.399602Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.399617Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 95 partNo 0 count 1 size 262204
2025-11-28T18:22:32.401449Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 95 count 1 size 262204 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.401469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.401486Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 95 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.401488Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.401496Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-28T18:22:32.401506Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 95, partNo: 0, Offset: 95 is stored on disk
2025-11-28T18:22:32.401540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.401546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.401548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.401550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.401551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.401561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000064_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000065_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000066_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000067_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000068_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000069_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000070_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000071_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000072_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000073_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000074_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000075_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000076_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000077_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000078_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000079_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000080_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000081_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000082_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000083_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000084_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000085_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000086_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000087_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000088_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000089_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000090_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000091_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000092_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000093_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000094_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000095_00000_0000000001_00000? (262204)
2025-11-28T18:22:32.401668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 32 keys were taken away. Let's read 8390528 bytes
2025-11-28T18:22:32.401682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.401703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 32 blobs
2025-11-28T18:22:32.401709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000064_00000_0000000001_00000?
2025-11-28T18:22:32.401718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000065_00000_0000000001_00000?
2025-11-28T18:22:32.401730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000066_00000_0000000001_00000?
2025-11-28T18:22:32.401743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000067_00000_0000000001_00000?
2025-11-28T18:22:32.401754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000068_00000_0000000001_00000?
2025-11-28T18:22:32.401773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000069_00000_0000000001_00000?
2025-11-28T18:22:32.401783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000070_00000_0000000001_00000?
2025-11-28T18:22:32.401791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000071_00000_0000000001_00000?
2025-11-28T18:22:32.401801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000072_00000_0000000001_00000?
2025-11-28T18:22:32.401777Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.401821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000073_00000_0000000001_00000?
2025-11-28T18:22:32.401830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000074_00000_0000000001_00000?
2025-11-28T18:22:32.401840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000075_00000_0000000001_00000?
2025-11-28T18:22:32.401851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000076_00000_0000000001_00000?
2025-11-28T18:22:32.401863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000077_00000_0000000001_00000?
2025-11-28T18:22:32.401873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000078_00000_0000000001_00000?
2025-11-28T18:22:32.401884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000079_00000_0000000001_00000?
2025-11-28T18:22:32.401895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000080_00000_0000000001_00000?
2025-11-28T18:22:32.401934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000081_00000_0000000001_00000?
2025-11-28T18:22:32.401945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000082_00000_0000000001_00000?
2025-11-28T18:22:32.401952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000083_00000_0000000001_00000?
2025-11-28T18:22:32.401963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000084_00000_0000000001_00000?
2025-11-28T18:22:32.401974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000085_00000_0000000001_00000?
2025-11-28T18:22:32.401985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000086_00000_0000000001_00000?
2025-11-28T18:22:32.401991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000087_00000_0000000001_00000?
2025-11-28T18:22:32.401998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000088_00000_0000000001_00000?
2025-11-28T18:22:32.402007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000089_00000_0000000001_00000?
2025-11-28T18:22:32.402018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000090_00000_0000000001_00000?
2025-11-28T18:22:32.402029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000091_00000_0000000001_00000?
2025-11-28T18:22:32.402036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000092_00000_0000000001_00000?
2025-11-28T18:22:32.402047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000093_00000_0000000001_00000?
2025-11-28T18:22:32.402058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000094_00000_0000000001_00000?
2025-11-28T18:22:32.402069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000095_00000_0000000001_00000?
2025-11-28T18:22:32.402077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 32 blobs for compaction
2025-11-28T18:22:32.402088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045291:3660] destroyed
2025-11-28T18:22:32.402103Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 64 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
CallPersQueueGRPC response:
2025-11-28T18:22:32.402111Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 65 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402114Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 66 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402116Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 67 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402119Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 68 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402121Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 69 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402125Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 70 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402127Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 71 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402130Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 72 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402132Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 73 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402135Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 74 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402137Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 75 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402139Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 76 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402142Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 77 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402145Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 78 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402147Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 79 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402151Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 80 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402157Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 81 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402160Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 82 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402162Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 83 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402166Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 84 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402169Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 85 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 95
    Offset: 95
    WriteTimestampMS: 1764354152398
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-28T18:22:32.402172Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 86 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402179Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 87 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402181Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 88 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402184Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 89 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402188Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 90 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402194Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 91 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402196Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 92 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402199Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 93 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402202Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 94 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.402208Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 95 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
CallPersQueueGRPC request to localhost:19639
2025-11-28T18:22:32.402222Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 32 blobs are from cache.
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdRead {
    ClientId: "user1"
    Offset: 0
    Count: 48
    ReadTimestampMs: 0
  }
}
2025-11-28T18:22:32.402242Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 64 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402253Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 65 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402256Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 66 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-28T18:22:32.402263Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 67 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/32] d0000000000_00000000000000000064_00000_0000000001_00000?
2025-11-28T18:22:32.402266Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 68 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.402270Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 69 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000064_00000_0000000001_00000?, parameters.CurOffset=64
2025-11-28T18:22:32.402273Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 70 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402276Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 71 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402281Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 72 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402284Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 73 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402288Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 74 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402291Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 75 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402294Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 76 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402297Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 77 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402299Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 78 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402302Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 79 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402305Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 80 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402307Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 81 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402310Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 82 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402312Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 83 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402315Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 84 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402318Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 85 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402322Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 86 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402324Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 87 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402327Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 88 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402330Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 89 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402343Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 90 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402351Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 91 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402353Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 92 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402356Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 93 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402360Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 94 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 64.0/1
2025-11-28T18:22:32.402375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 64 partNo 0
2025-11-28T18:22:32.402381Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 95 partno 0 count 1 parts 0 suffix '63'
2025-11-28T18:22:32.402393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 64 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 262214 count 1 nextOffset 65 batches 1
2025-11-28T18:22:32.402396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 64:0 appended
2025-11-28T18:22:32.402399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/32] d0000000000_00000000000000000065_00000_0000000001_00000?
2025-11-28T18:22:32.402400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.402402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000065_00000_0000000001_00000?, parameters.CurOffset=65
2025-11-28T18:22:32.402505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 65.0/1
2025-11-28T18:22:32.402516Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 65 partNo 0
2025-11-28T18:22:32.402689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 65 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 524387 count 2 nextOffset 66 batches 1
2025-11-28T18:22:32.402700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 65:0 appended
2025-11-28T18:22:32.402704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/32] d0000000000_00000000000000000066_00000_0000000001_00000?
2025-11-28T18:22:32.402707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.402709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000066_00000_0000000001_00000?, parameters.CurOffset=66
2025-11-28T18:22:32.402746Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.402779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 66.0/1
2025-11-28T18:22:32.402788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 66 partNo 0
2025-11-28T18:22:32.402799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 66 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 786601 count 3 nextOffset 67 batches 2
2025-11-28T18:22:32.402805Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 66:0 appended
2025-11-28T18:22:32.402808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/32] d0000000000_00000000000000000067_00000_0000000001_00000?
2025-11-28T18:22:32.402810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.402811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000067_00000_0000000001_00000?, parameters.CurOffset=67
2025-11-28T18:22:32.402873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 67.0/1
2025-11-28T18:22:32.402880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 67 partNo 0
2025-11-28T18:22:32.403030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 67 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 1048774 count 4 nextOffset 68 batches 2
2025-11-28T18:22:32.403039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 67:0 appended
2025-11-28T18:22:32.403043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/32] d0000000000_00000000000000000068_00000_0000000001_00000?
2025-11-28T18:22:32.403045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.403046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000068_00000_0000000001_00000?, parameters.CurOffset=68
2025-11-28T18:22:32.403113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 68.0/1
2025-11-28T18:22:32.403125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 68 partNo 0
2025-11-28T18:22:32.403134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 68 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 1310988 count 5 nextOffset 69 batches 3
2025-11-28T18:22:32.403136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 68:0 appended
2025-11-28T18:22:32.403139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/32] d0000000000_00000000000000000069_00000_0000000001_00000?
2025-11-28T18:22:32.403141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.403142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000069_00000_0000000001_00000?, parameters.CurOffset=69
2025-11-28T18:22:32.403204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 69.0/1
2025-11-28T18:22:32.403213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 69 partNo 0
2025-11-28T18:22:32.403342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 69 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 1573161 count 6 nextOffset 70 batches 3
2025-11-28T18:22:32.403352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 69:0 appended
2025-11-28T18:22:32.403355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/32] d0000000000_00000000000000000070_00000_0000000001_00000?
2025-11-28T18:22:32.403356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.403360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000070_00000_0000000001_00000?, parameters.CurOffset=70
2025-11-28T18:22:32.403427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 70.0/1
2025-11-28T18:22:32.403434Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 70 partNo 0
2025-11-28T18:22:32.403455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 70 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 1835375 count 7 nextOffset 71 batches 4
2025-11-28T18:22:32.403460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 70:0 appended
2025-11-28T18:22:32.403462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/32] d0000000000_00000000000000000071_00000_0000000001_00000?
2025-11-28T18:22:32.403464Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.403466Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000071_00000_0000000001_00000?, parameters.CurOffset=71
2025-11-28T18:22:32.403531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 71.0/1
2025-11-28T18:22:32.403538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 71 partNo 0
2025-11-28T18:22:32.403661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 71 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 2097548 count 8 nextOffset 72 batches 4
2025-11-28T18:22:32.403670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 71:0 appended
2025-11-28T18:22:32.403674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[8/32] d0000000000_00000000000000000072_00000_0000000001_00000?
2025-11-28T18:22:32.403676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.403678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000072_00000_0000000001_00000?, parameters.CurOffset=72
2025-11-28T18:22:32.403746Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 72.0/1
2025-11-28T18:22:32.403756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 72 partNo 0
2025-11-28T18:22:32.403765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 72 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 2359762 count 9 nextOffset 73 batches 5
2025-11-28T18:22:32.403770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 72:0 appended
2025-11-28T18:22:32.403773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[9/32] d0000000000_00000000000000000073_00000_0000000001_00000?
2025-11-28T18:22:32.403774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.403776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000073_00000_0000000001_00000?, parameters.CurOffset=73
2025-11-28T18:22:32.403844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 73.0/1
2025-11-28T18:22:32.403851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 73 partNo 0
2025-11-28T18:22:32.403982Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 73 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 2621935 count 10 nextOffset 74 batches 5
2025-11-28T18:22:32.403990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 73:0 appended
2025-11-28T18:22:32.403994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[10/32] d0000000000_00000000000000000074_00000_0000000001_00000?
2025-11-28T18:22:32.403995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.403997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000074_00000_0000000001_00000?, parameters.CurOffset=74
2025-11-28T18:22:32.404069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 74.0/1
2025-11-28T18:22:32.404077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 74 partNo 0
2025-11-28T18:22:32.404086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 74 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 2884149 count 11 nextOffset 75 batches 6
2025-11-28T18:22:32.404093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 74:0 appended
2025-11-28T18:22:32.404096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[11/32] d0000000000_00000000000000000075_00000_0000000001_00000?
2025-11-28T18:22:32.404098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.404099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000075_00000_0000000001_00000?, parameters.CurOffset=75
2025-11-28T18:22:32.404163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 75.0/1
2025-11-28T18:22:32.404171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 75 partNo 0
2025-11-28T18:22:32.404336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 75 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 3146322 count 12 nextOffset 76 batches 6
2025-11-28T18:22:32.404344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 75:0 appended
2025-11-28T18:22:32.404347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[12/32] d0000000000_00000000000000000076_00000_0000000001_00000?
2025-11-28T18:22:32.404348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.404350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000076_00000_0000000001_00000?, parameters.CurOffset=76
2025-11-28T18:22:32.404418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 76.0/1
2025-11-28T18:22:32.404428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 76 partNo 0
2025-11-28T18:22:32.404435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 76 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 3408536 count 13 nextOffset 77 batches 7
2025-11-28T18:22:32.404437Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 76:0 appended
2025-11-28T18:22:32.404440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[13/32] d0000000000_00000000000000000077_00000_0000000001_00000?
2025-11-28T18:22:32.404441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.404443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000077_00000_0000000001_00000?, parameters.CurOffset=77
2025-11-28T18:22:32.404500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 77.0/1
2025-11-28T18:22:32.404507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 77 partNo 0
2025-11-28T18:22:32.404617Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 77 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 3670709 count 14 nextOffset 78 batches 7
2025-11-28T18:22:32.404625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 77:0 appended
2025-11-28T18:22:32.404629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[14/32] d0000000000_00000000000000000078_00000_0000000001_00000?
2025-11-28T18:22:32.404630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.404632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000078_00000_0000000001_00000?, parameters.CurOffset=78
2025-11-28T18:22:32.404698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 78.0/1
2025-11-28T18:22:32.404706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 78 partNo 0
2025-11-28T18:22:32.404718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 78 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 3932923 count 15 nextOffset 79 batches 8
2025-11-28T18:22:32.404737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 78:0 appended
2025-11-28T18:22:32.404741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[15/32] d0000000000_00000000000000000079_00000_0000000001_00000?
2025-11-28T18:22:32.404742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.404744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000079_00000_0000000001_00000?, parameters.CurOffset=79
2025-11-28T18:22:32.404809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 79.0/1
2025-11-28T18:22:32.404816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 79 partNo 0
2025-11-28T18:22:32.404932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 79 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 4195096 count 16 nextOffset 80 batches 8
2025-11-28T18:22:32.404944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 79:0 appended
2025-11-28T18:22:32.404948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[16/32] d0000000000_00000000000000000080_00000_0000000001_00000?
2025-11-28T18:22:32.404949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.404950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000080_00000_0000000001_00000?, parameters.CurOffset=80
2025-11-28T18:22:32.405017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 80.0/1
2025-11-28T18:22:32.405021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 80 partNo 0
2025-11-28T18:22:32.405029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 80 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 4457310 count 17 nextOffset 81 batches 9
2025-11-28T18:22:32.405032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 80:0 appended
2025-11-28T18:22:32.405034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[17/32] d0000000000_00000000000000000081_00000_0000000001_00000?
2025-11-28T18:22:32.405035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.405037Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000081_00000_0000000001_00000?, parameters.CurOffset=81
2025-11-28T18:22:32.405100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 81.0/1
2025-11-28T18:22:32.405102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 81 partNo 0
2025-11-28T18:22:32.405222Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 81 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 4719483 count 18 nextOffset 82 batches 9
2025-11-28T18:22:32.405225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 81:0 appended
2025-11-28T18:22:32.405228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[18/32] d0000000000_00000000000000000082_00000_0000000001_00000?
2025-11-28T18:22:32.405230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.405232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000082_00000_0000000001_00000?, parameters.CurOffset=82
2025-11-28T18:22:32.405291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 82.0/1
2025-11-28T18:22:32.405294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 82 partNo 0
2025-11-28T18:22:32.405312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 82 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 4981697 count 19 nextOffset 83 batches 10
2025-11-28T18:22:32.405314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 82:0 appended
2025-11-28T18:22:32.405317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[19/32] d0000000000_00000000000000000083_00000_0000000001_00000?
2025-11-28T18:22:32.405318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.405321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000083_00000_0000000001_00000?, parameters.CurOffset=83
2025-11-28T18:22:32.405372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 83.0/1
2025-11-28T18:22:32.405374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 83 partNo 0
2025-11-28T18:22:32.405486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 83 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 5243870 count 20 nextOffset 84 batches 10
2025-11-28T18:22:32.405488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 83:0 appended
2025-11-28T18:22:32.405493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[20/32] d0000000000_00000000000000000084_00000_0000000001_00000?
2025-11-28T18:22:32.405495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.405497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000084_00000_0000000001_00000?, parameters.CurOffset=84
2025-11-28T18:22:32.405555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 84.0/1
2025-11-28T18:22:32.405558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 84 partNo 0
2025-11-28T18:22:32.405625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 84 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 5506084 count 21 nextOffset 85 batches 11
2025-11-28T18:22:32.405633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 84:0 appended
2025-11-28T18:22:32.405640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[21/32] d0000000000_00000000000000000085_00000_0000000001_00000?
2025-11-28T18:22:32.405642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.405644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000085_00000_0000000001_00000?, parameters.CurOffset=85
2025-11-28T18:22:32.405708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 85.0/1
2025-11-28T18:22:32.405721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 85 partNo 0
2025-11-28T18:22:32.406219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 85 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 5768257 count 22 nextOffset 86 batches 11
2025-11-28T18:22:32.406232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 85:0 appended
2025-11-28T18:22:32.406236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[22/32] d0000000000_00000000000000000086_00000_0000000001_00000?
2025-11-28T18:22:32.406238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.406240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000086_00000_0000000001_00000?, parameters.CurOffset=86
2025-11-28T18:22:32.406303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 86.0/1
2025-11-28T18:22:32.406312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 86 partNo 0
2025-11-28T18:22:32.406325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 86 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 6030471 count 23 nextOffset 87 batches 12
2025-11-28T18:22:32.406330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 86:0 appended
2025-11-28T18:22:32.406333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[23/32] d0000000000_00000000000000000087_00000_0000000001_00000?
2025-11-28T18:22:32.406334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.406336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000087_00000_0000000001_00000?, parameters.CurOffset=87
2025-11-28T18:22:32.406386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 87.0/1
2025-11-28T18:22:32.406393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 87 partNo 0
2025-11-28T18:22:32.406881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 87 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 6292644 count 24 nextOffset 88 batches 12
2025-11-28T18:22:32.406894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 87:0 appended
2025-11-28T18:22:32.406898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[24/32] d0000000000_00000000000000000088_00000_0000000001_00000?
2025-11-28T18:22:32.406899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.406901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000088_00000_0000000001_00000?, parameters.CurOffset=88
2025-11-28T18:22:32.406965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 88.0/1
2025-11-28T18:22:32.406977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 88 partNo 0
2025-11-28T18:22:32.406987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 88 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 6554858 count 25 nextOffset 89 batches 13
2025-11-28T18:22:32.406992Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 88:0 appended
2025-11-28T18:22:32.406995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[25/32] d0000000000_00000000000000000089_00000_0000000001_00000?
2025-11-28T18:22:32.406996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.406998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000089_00000_0000000001_00000?, parameters.CurOffset=89
2025-11-28T18:22:32.407049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 89.0/1
2025-11-28T18:22:32.407057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 89 partNo 0
2025-11-28T18:22:32.407534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 89 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 6817031 count 26 nextOffset 90 batches 13
2025-11-28T18:22:32.407545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 89:0 appended
2025-11-28T18:22:32.407549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[26/32] d0000000000_00000000000000000090_00000_0000000001_00000?
2025-11-28T18:22:32.407551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.407553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000090_00000_0000000001_00000?, parameters.CurOffset=90
2025-11-28T18:22:32.407613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 90.0/1
2025-11-28T18:22:32.407622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 90 partNo 0
2025-11-28T18:22:32.407632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 90 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 7079245 count 27 nextOffset 91 batches 14
2025-11-28T18:22:32.407637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 90:0 appended
2025-11-28T18:22:32.407640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[27/32] d0000000000_00000000000000000091_00000_0000000001_00000?
2025-11-28T18:22:32.407642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.407643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000091_00000_0000000001_00000?, parameters.CurOffset=91
2025-11-28T18:22:32.407690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 91.0/1
2025-11-28T18:22:32.407697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 91 partNo 0
2025-11-28T18:22:32.408147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 91 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 7341418 count 28 nextOffset 92 batches 14
2025-11-28T18:22:32.408160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 91:0 appended
2025-11-28T18:22:32.408164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[28/32] d0000000000_00000000000000000092_00000_0000000001_00000?
2025-11-28T18:22:32.408165Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.408167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000092_00000_0000000001_00000?, parameters.CurOffset=92
2025-11-28T18:22:32.408220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 92.0/1
2025-11-28T18:22:32.408227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 92 partNo 0
2025-11-28T18:22:32.408238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 92 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 7603632 count 29 nextOffset 93 batches 15
2025-11-28T18:22:32.408244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 92:0 appended
2025-11-28T18:22:32.408246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[29/32] d0000000000_00000000000000000093_00000_0000000001_00000?
2025-11-28T18:22:32.408250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.408252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000093_00000_0000000001_00000?, parameters.CurOffset=93
2025-11-28T18:22:32.408307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 93.0/1
2025-11-28T18:22:32.408314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 93 partNo 0
2025-11-28T18:22:32.414212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 93 partNo 0 result is x0000000000_00000000000000000062_00000_0000000031_00000 size 8128009
2025-11-28T18:22:32.414231Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1110: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000062_00000_0000000031_00000 new key d0000000000_00000000000000000062_00000_0000000031_00000 size 8128009 WTime 1764354152414
2025-11-28T18:22:32.414258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 93 partNo 0 FormedBlobsCount 1 NewHead: Offset 93 PartNo 0 PackedSize 262214 count 1 nextOffset 94 batches 1
2025-11-28T18:22:32.414262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 93:0 appended
2025-11-28T18:22:32.414266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[30/32] d0000000000_00000000000000000094_00000_0000000001_00000?
2025-11-28T18:22:32.414268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.414270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000094_00000_0000000001_00000?, parameters.CurOffset=94
2025-11-28T18:22:32.414348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 94.0/1
2025-11-28T18:22:32.414355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 94 partNo 0
2025-11-28T18:22:32.414509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 94 partNo 0 FormedBlobsCount 0 NewHead: Offset 93 PartNo 0 PackedSize 524387 count 2 nextOffset 95 batches 1
2025-11-28T18:22:32.414515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 94:0 appended
2025-11-28T18:22:32.414519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[31/32] d0000000000_00000000000000000095_00000_0000000001_00000?
2025-11-28T18:22:32.414520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-28T18:22:32.414521Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000095_00000_0000000001_00000?, parameters.CurOffset=95
2025-11-28T18:22:32.414583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 95.0/1
2025-11-28T18:22:32.414595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 95 partNo 0
2025-11-28T18:22:32.414606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 95 partNo 0 FormedBlobsCount 0 NewHead: Offset 93 PartNo 0 PackedSize 786601 count 3 nextOffset 96 batches 2
2025-11-28T18:22:32.414610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 95:0 appended
2025-11-28T18:22:32.414678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 93,3 HeadOffset 62 endOffset 64 curOffset 96 d0000000000_00000000000000000093_00000_0000000003_00000| size 786591 WTime 1764354152414
2025-11-28T18:22:32.414990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045301:3666], now have 1 active actors on pipe
2025-11-28T18:22:32.415043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.415054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.415074Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.415098Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 62 partNo 0 count 31 size 8128009
2025-11-28T18:22:32.415108Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 93 partNo 0 count 3 size 786591
2025-11-28T18:22:32.415477Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 4 Topic 'rt3.dc1--topic1' partition 0 user user1 offset 0 partno 0 count 48 size 26214400 endOffset 96 max time lag 0ms effective offset 0
2025-11-28T18:22:32.415665Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 4 added 2 blobs, size 16256018 count 63 last offset 31, current partition end offset: 96
2025-11-28T18:22:32.415676Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 4. Send blob request.
2025-11-28T18:22:32.415692Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 31 parts_count 0 source 1 size 8128009 accessed 0 times before, last time 2025-11-28T18:22:31.000000Z
2025-11-28T18:22:32.415701Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 31 partno 0 count 31 parts_count 0 source 1 size 8128009 accessed 0 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.415722Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 4. All 2 blobs are from cache.
2025-11-28T18:22:32.415745Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 31 parts 0 suffix '0'
2025-11-28T18:22:32.415755Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 31 parts 0 suffix '0'
2025-11-28T18:22:32.415756Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 2 blobs
2025-11-28T18:22:32.415967Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.416287Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 2 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.416740Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 4 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.417223Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 6 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.417597Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 8 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.418107Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 10 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.418631Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 12 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.419209Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 14 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.419742Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 16 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.420250Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 18 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.420796Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 20 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.421330Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 22 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.421716Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 24 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.422078Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 26 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.422426Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 28 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.422690Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 30 totakecount 31 count 1 size 262182 from pos 0 cbcount 1
2025-11-28T18:22:32.422799Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 31 totakecount 31 count 1 size 262182 from pos 0 cbcount 1
2025-11-28T18:22:32.422966Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 32 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.423303Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 34 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.423653Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 36 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.424001Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 38 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.424369Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 40 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.424737Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 42 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.425264Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 44 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.425723Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 46 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.425811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.438441Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 62 count 31 size 8128009 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.438513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 93 count 3 size 786591 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.438537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.438553Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 31 parts 0 suffix '0' size 8128009
2025-11-28T18:22:32.438562Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 2 parts 0 suffix '124' size 524387
2025-11-28T18:22:32.438566Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 64 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.438569Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 93 partno 0 count 3 parts 0 suffix '124' size 786591
2025-11-28T18:22:32.438720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.438732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.438734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.438738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.438740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.438789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.438805Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 64 size 262204 cause it's been evicted from L2. Actual L1 size: 36
2025-11-28T18:22:32.438807Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 62 size 524387 cause it's been evicted from L2. Actual L1 size: 35
2025-11-28T18:22:32.438812Z node 2 :PERSQUEUE NOTICE: read.h:372: [72075186224037892][PQCacheProxy]Have to remove new data from cache. Topic rt3.dc1--topic1, tablet id72075186224037892, cookie 0
2025-11-28T18:22:32.438818Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-28T18:22:32.438828Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000064_00000_0000000001_00000?(+) to d0000000000_00000000000000000064_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438830Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000065_00000_0000000001_00000?(+) to d0000000000_00000000000000000065_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438832Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000066_00000_0000000001_00000?(+) to d0000000000_00000000000000000066_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438834Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000067_00000_0000000001_00000?(+) to d0000000000_00000000000000000067_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438836Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000068_00000_0000000001_00000?(+) to d0000000000_00000000000000000068_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438837Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000069_00000_0000000001_00000?(+) to d0000000000_00000000000000000069_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438839Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000070_00000_0000000001_00000?(+) to d0000000000_00000000000000000070_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438841Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000071_00000_0000000001_00000?(+) to d0000000000_00000000000000000071_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438842Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000072_00000_0000000001_00000?(+) to d0000000000_00000000000000000072_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438844Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000073_00000_0000000001_00000?(+) to d0000000000_00000000000000000073_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438846Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000074_00000_0000000001_00000?(+) to d0000000000_00000000000000000074_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438847Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000075_00000_0000000001_00000?(+) to d0000000000_00000000000000000075_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438849Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000076_00000_0000000001_00000?(+) to d0000000000_00000000000000000076_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438850Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000077_00000_0000000001_00000?(+) to d0000000000_00000000000000000077_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438851Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000078_00000_0000000001_00000?(+) to d0000000000_00000000000000000078_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438853Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000079_00000_0000000001_00000?(+) to d0000000000_00000000000000000079_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438855Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000080_00000_0000000001_00000?(+) to d0000000000_00000000000000000080_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438858Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000081_00000_0000000001_00000?(+) to d0000000000_00000000000000000081_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438860Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000082_00000_0000000001_00000?(+) to d0000000000_00000000000000000082_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438862Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000083_00000_0000000001_00000?(+) to d0000000000_00000000000000000083_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438864Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000084_00000_0000000001_00000?(+) to d0000000000_00000000000000000084_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438867Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000085_00000_0000000001_00000?(+) to d0000000000_00000000000000000085_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438869Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000086_00000_0000000001_00000?(+) to d0000000000_00000000000000000086_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438883Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000087_00000_0000000001_00000?(+) to d0000000000_00000000000000000087_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438885Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000088_00000_0000000001_00000?(+) to d0000000000_00000000000000000088_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438887Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000089_00000_0000000001_00000?(+) to d0000000000_00000000000000000089_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438923Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000090_00000_0000000001_00000?(+) to d0000000000_00000000000000000090_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438924Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000091_00000_0000000001_00000?(+) to d0000000000_00000000000000000091_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438926Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000092_00000_0000000001_00000?(+) to d0000000000_00000000000000000092_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438928Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000093_00000_0000000001_00000?(+) to d0000000000_00000000000000000093_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438929Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000094_00000_0000000001_00000?(+) to d0000000000_00000000000000000094_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438930Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000095_00000_0000000001_00000?(+) to d0000000000_00000000000000000095_00000_0000000001_00000?(+)
2025-11-28T18:22:32.438932Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000062_00000_0000000002_00000|(+) to d0000000000_00000000000000000062_00000_0000000002_00000|(+)
2025-11-28T18:22:32.442175Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.440141Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 65 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440154Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 66 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440156Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 67 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440159Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 68 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440163Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 69 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440166Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 70 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440168Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 71 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440170Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 72 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440173Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 73 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440176Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 74 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440179Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 75 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440181Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 76 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440184Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 77 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440187Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 78 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440189Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 79 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440191Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 80 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440195Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 81 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440197Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 82 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440200Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 83 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440202Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 84 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440204Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 85 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440207Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 86 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440209Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 87 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440211Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 88 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 89 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440216Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 90 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440219Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 91 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440221Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 92 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440223Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 93 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440225Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 94 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 95 count 1 actorID [2:7577843380023719316:2363]
2025-11-28T18:22:32.440248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-28T18:22:32.440258Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 65 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440263Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 66 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440264Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:576: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-28T18:22:32.440267Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 67 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440269Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 68 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440272Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 69 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440274Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 70 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.440278Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 71 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.440281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.440281Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 72 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.440284Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 73 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.440287Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 74 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440289Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 75 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-28T18:22:32.440292Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 76 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440296Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 77 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440299Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 78 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440301Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 79 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440305Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 80 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440308Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 81 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440310Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 82 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440323Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 83 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440326Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 84 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440329Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 85 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440333Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 86 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440336Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 87 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440340Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 88 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440342Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 89 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440344Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 90 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440347Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 91 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440350Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 92 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440353Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 93 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440355Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 94 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.440358Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 95 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-28T18:22:32.446231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.446243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.446245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.446247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.446248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.450161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045301:3666] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdReadResult {
    MaxOffset: 96
    Result {
      Offset: 0
      Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
CallPersQueueGRPC request to localhost:19639
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdRead {
    ClientId: "user1"
    Offset: 0
    Count: 48
    ReadTimestampMs: 0
  }
}
2025-11-28T18:22:32.502105Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-28T18:22:32.502654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577843385349045347:3686], now have 1 active actors on pipe
2025-11-28T18:22:32.502800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-28T18:22:32.502812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-28T18:22:32.502849Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 5 Topic 'rt3.dc1--topic1' partition 0 user user1 offset 0 partno 0 count 48 size 26214400 endOffset 96 max time lag 0ms effective offset 0
2025-11-28T18:22:32.502877Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 5 added 2 blobs, size 16256018 count 62 last offset 31, current partition end offset: 96
2025-11-28T18:22:32.502883Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 5. Send blob request.
2025-11-28T18:22:32.502893Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 31 parts_count 0 source 1 size 8128009 accessed 1 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.502899Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 31 partno 0 count 31 parts_count 0 source 1 size 8128009 accessed 1 times before, last time 2025-11-28T18:22:32.000000Z
2025-11-28T18:22:32.502910Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 5. All 2 blobs are from cache.
2025-11-28T18:22:32.502930Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 31 parts 0 suffix '0'
2025-11-28T18:22:32.502938Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 31 parts 0 suffix '0'
2025-11-28T18:22:32.502946Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 2 blobs
2025-11-28T18:22:32.503159Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.503325Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 2 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.503637Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 4 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.503966Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 6 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.504154Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 8 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.504343Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 10 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.504520Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 12 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.504699Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 14 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.504890Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 16 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.505265Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 18 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.505599Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 20 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.505939Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 22 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.506256Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 24 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.506418Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 26 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.506613Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 28 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.506678Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 30 totakecount 31 count 1 size 262182 from pos 0 cbcount 1
2025-11-28T18:22:32.506760Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 31 totakecount 31 count 1 size 262182 from pos 0 cbcount 1
2025-11-28T18:22:32.506897Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 32 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.507055Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 34 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.507212Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 36 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.507373Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 38 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.507530Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 40 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.507689Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 42 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.507848Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 44 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.508005Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 46 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-28T18:22:32.508082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-28T18:22:32.522777Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-28T18:22:32.528536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577843385349045347:3686] destroyed
CallPersQueueGRPC response:
2025-11-28T18:22:32.546618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.546637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.546640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.546645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.546647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdReadResult {
    MaxOffset: 96
    Result {
      Offset: 0
      Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-28T18:22:32.646992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.647009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.647012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.647015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.647018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-28T18:22:32.747336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-28T18:22:32.747359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.747362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-28T18:22:32.747366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-28T18:22:32.747369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
[[bad]]assertion failed at ydb/services/persqueue_v1/persqueue_ut.cpp:3344, virtual void NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TTestCaseCacheHead::Execute_(NUnitTest::TTestContext &): (info_half1.BlobsFromDisk == 1) failed: (0 != 1) [[rst]]
[[alt1]]TBackTrace::Capture()+28 (0x15100E9C)
NUnitTest::NPrivate::RaiseError(char const*, TBasicString<char, std::__y1::char_traits<char>> const&, bool)+128 (0x152BCEA0)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TTestCaseCacheHead::Execute_(NUnitTest::TTestContext&)+5702 (0x14E35A66)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TCurrentTest::Execute()::'lambda'()::operator()() const+71 (0x14F17D07)
NUnitTest::TTestBase::Run(std::__y1::function<void ()>, TBasicString<char, std::__y1::char_traits<char>> const&, char const*, bool)+120 (0x152BF078)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TCurrentTest::Execute()+436 (0x14F17604)
NUnitTest::TTestFactory::Execute()+817 (0x152BF7C1)
NUnitTest::RunMain(int, char**)+3229 (0x152D1C2D)
??+0 (0x7FFBAAFF1D90)
__libc_start_main+128 (0x7FFBAAFF1E40)
_start+41 (0x13F1F029)
[[rst]]
