2025-11-30T08:30:12.630075Z node 1 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[1:7578432911914950443:2240];send_to=[0:7307199536658146131:7762515];
2025-11-30T08:30:12.630097Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
2025-11-30T08:30:12.638800Z node 2 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[2:7578432910247620121:2078];send_to=[0:7307199536658146131:7762515];
2025-11-30T08:30:12.638900Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
test_client.cpp: SetPath # /home/runner/.ya/build/build_root/0ao2/000a16/r3tmp/tmpDC3U8S/pdisk_1.dat
2025-11-30T08:30:12.640059Z node 1 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-11-30T08:30:12.640903Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-11-30T08:30:12.683434Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-11-30T08:30:12.683926Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-11-30T08:30:12.700478Z node 1 :IMPORT WARN: schemeshard_import.cpp:423: Table profiles were not loaded
TServer::EnableGrpc on GrpcPort 12447, node 1
2025-11-30T08:30:12.731033Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-11-30T08:30:12.731087Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-11-30T08:30:12.733247Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-11-30T08:30:12.743471Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-11-30T08:30:12.743504Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-11-30T08:30:12.745485Z node 1 :HIVE WARN: hive_impl.cpp:811: HIVE#72057594037968897 Handle TEvInterconnect::TEvNodeConnected, NodeId 2 Cookie 2
2025-11-30T08:30:12.745616Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-11-30T08:30:12.774002Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:190: distributable config is empty, broken or outdated, will use file: /home/runner/.ya/build/build_root/0ao2/000a16/r3tmp/yandexmDzHwv.tmp
2025-11-30T08:30:12.774019Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:196: will try to initialize from file: /home/runner/.ya/build/build_root/0ao2/000a16/r3tmp/yandexmDzHwv.tmp
2025-11-30T08:30:12.774085Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:202: successfully initialized from file: /home/runner/.ya/build/build_root/0ao2/000a16/r3tmp/yandexmDzHwv.tmp
2025-11-30T08:30:12.774128Z node 1 :NET_CLASSIFIER ERROR: net_classifier.cpp:228: got bad distributable configuration
2025-11-30T08:30:12.838834Z INFO: TTestServer started on Port 63009 GrpcPort 12447
TClient is connected to server localhost:63009
PQClient connected to localhost:12447
WaitRootIsUp 'Root'...
TClient::Ls request: Root
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "Root"
    PathId: 1
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 1
    CreateStep: 0
    ParentPathId: 1
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 1
      SubDomainVersion: 0
      SecurityStateVersion: 0
    }
    ChildrenExist: false
  }
  Children {
    Name: ".sys"
    PathId: 18446744073709551615
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 0
    CreateStep: 0
    ParentPathId: 18446744073709551615
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 0
      Pl...
(TRUNCATED)
WaitRootIsUp 'Root' success.
2025-11-30T08:30:12.877395Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpAlterSubDomain, opId: 281474976720657:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_alter_subdomain.cpp:311)
2025-11-30T08:30:12.887016Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
waiting...
2025-11-30T08:30:12.896571Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
waiting...
2025-11-30T08:30:12.921141Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__notify.cpp:122: NotifyTxCompletion, unknown transaction, txId: 281474976720658, at schemeshard: 72057594046644480
2025-11-30T08:30:12.927140Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720659:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
waiting...
waiting...
2025-11-30T08:30:13.037099Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__notify.cpp:122: NotifyTxCompletion, unknown transaction, txId: 281474976720660, at schemeshard: 72057594046644480
waiting...
2025-11-30T08:30:13.078109Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7578432916209918555:2326], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-30T08:30:13.078158Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-30T08:30:13.078784Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7578432916209918567:2329], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-30T08:30:13.078799Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7578432916209918568:2330], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-30T08:30:13.078817Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-30T08:30:13.078271Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7578432914542587782:2300], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-30T08:30:13.078302Z node 2 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-30T08:30:13.078363Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7578432914542587793:2303], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-30T08:30:13.078709Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7578432914542587796:2304], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-30T08:30:13.078719Z node 2 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-30T08:30:13.082140Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateResourcePool, opId: 281474976720662:3, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:179)
2025-11-30T08:30:13.084191Z node 2 :TX_PROXY ERROR: schemereq.cpp:593: Actor# [2:7578432914542587799:2165] txid# 281474976715657, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exists but creating right now (id: [OwnerId: 72057594046644480, LocalPathId: 9], type: EPathTypeResourcePool, state: EPathStateCreate), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-11-30T08:30:13.090828Z node 1 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [1:7578432916209918571:2331], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976720662 completed, doublechecking }
2025-11-30T08:30:13.091065Z node 2 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [2:7578432914542587797:2305], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976720662 completed, doublechecking }
2025-11-30T08:30:13.157589Z node 1 :TX_PROXY ERROR: schemereq.cpp:593: Actor# [1:7578432916209918655:2722] txid# 281474976720663, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 9], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-11-30T08:30:13.158209Z node 2 :TX_PROXY ERROR: schemereq.cpp:593: Actor# [2:7578432914542587825:2171] txid# 281474976715658, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 9], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-11-30T08:30:13.241176Z node 1 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:625: Compilation failed, self: [1:7578432916209918669:2337], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-11-30T08:30:13.241350Z node 2 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:625: Compilation failed, self: [2:7578432914542587839:2309], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-11-30T08:30:13.241551Z node 2 :KQP_SESSION WARN: kqp_session_actor.cpp:2542: SessionId: ydb://session/3?node_id=2&id=ODFkODczZTEtOGI3NGI0ZWEtNTViY2M4OGEtNDk4OWQ0MzI=, ActorId: [2:7578432914542587780:2299], ActorState: ExecuteState, TraceId: 01kb9y4hjk792mzgyv9e8wzbbp, ReplyQueryCompileError, status: SCHEME_ERROR, issues: { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } }, remove tx with tx_id: 
2025-11-30T08:30:13.242085Z node 2 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-11-30T08:30:13.245060Z node 1 :KQP_SESSION WARN: kqp_session_actor.cpp:2542: SessionId: ydb://session/3?node_id=1&id=N2MzMzIyMjItNTRiMDk1NDgtZmU5ZTZiYWYtMzVjNmJiOTg=, ActorId: [1:7578432916209918552:2324], ActorState: ExecuteState, TraceId: 01kb9y4hjj2st9dpa3rvg46bn2, ReplyQueryCompileError, status: SCHEME_ERROR, issues: { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } }, remove tx with tx_id: 
2025-11-30T08:30:13.245213Z node 1 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-11-30T08:30:13.264838Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720664:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
2025-11-30T08:30:13.289020Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720665:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
2025-11-30T08:30:13.321932Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720666:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
=== Init DC: UPSERT INTO `/Root/PQ/Config/V2/Cluster` (name, balancer, local, enabled, weight) VALUES ("dc1", "localhost", true, true, 1000), ("dc2", "dc2.logbroker.yandex.net", false, true, 1000);
=== CheckClustersList. Subcribe to ClusterTracker from [1:7578432916209919096:3045] 
2025-11-30T08:30:13.630427Z node 1 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-11-30T08:30:13.642124Z node 2 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-11-30T08:30:17.630323Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[1:7578432911914950443:2240];send_to=[0:7307199536658146131:7762515];
2025-11-30T08:30:17.630374Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
2025-11-30T08:30:17.639289Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[2:7578432910247620121:2078];send_to=[0:7307199536658146131:7762515];
2025-11-30T08:30:17.639324Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
=== CheckClustersList. Ok
PQ Client: create topic: rt3.dc1--topic1 with 1 partitions
CallPersQueueGRPC request to localhost:12447
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--topic1"
  }
}
2025-11-30T08:30:18.568666Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--topic1, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:12447
MetaRequest {
  CmdCreateTopic {
    Topic: "rt3.dc1--topic1"
    NumPartitions: 1
    Config {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
  }
}
CallPersQueueGRPC response:
Status: 129
ProxyErrorCode: 53
SchemeStatus: 1
FlatTxId {
  TxId: 281474976720673
  SchemeShardTabletId: 72057594046644480
  PathId: 13
}
ErrorCode: OK
AddTopic: rt3.dc1--topic1
===Run query:``DECLARE $version as Int64; DECLARE $path AS Utf8; DECLARE $cluster as Utf8; UPSERT INTO `/Root/PQ/Config/V2/Topics` (path, dc) VALUES ($path, $cluster); UPSERT INTO `/Root/PQ/Config/V2/Versions` (name, version) VALUES ("Topics", $version);`` with topic = topic1, dc = dc1
2025-11-30T08:30:18.601890Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1656: [72075186224037893][] pipe [1:7578432937684756016:3287] connected; active server actors: 1
2025-11-30T08:30:18.602112Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1520: [72075186224037893][rt3.dc1--topic1] updating configuration. Deleted partitions []. Added partitions [0]
2025-11-30T08:30:18.602329Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:942: [72075186224037893][rt3.dc1--topic1] Discovered subdomain [OwnerId: 72057594046644480, LocalPathId: 1] state, outOfSpace = 0 at RB 72075186224037893
2025-11-30T08:30:18.607130Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:140: [72075186224037893][rt3.dc1--topic1] BALANCER INIT DONE for rt3.dc1--topic1: (0, 72075186224037892) 
2025-11-30T08:30:18.607637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2959: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-11-30T08:30:18.613701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2991: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-11-30T08:30:18.613812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:787: [PQ: 72075186224037892] no config, start with empty partitions and default config
2025-11-30T08:30:18.613829Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:913: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-11-30T08:30:18.613843Z node 2 :PERSQUEUE INFO: pq_impl.cpp:612: [PQ: 72075186224037892] doesn't have tx writes info
2025-11-30T08:30:18.614146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756034:2455], now have 1 active actors on pipe
2025-11-30T08:30:18.614154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756015:3286], now have 1 active actors on pipe
2025-11-30T08:30:18.614449Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:404: [72075186224037893][rt3.dc1--topic1] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1
2025-11-30T08:30:18.619311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3563: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-30T08:30:18.620680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-30T08:30:18.623091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756070:3331], now have 1 active actors on pipe
2025-11-30T08:30:18.624884Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:85: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitConfigStep
2025-11-30T08:30:18.624913Z node 2 :PERSQUEUE DEBUG: partition.cpp:1313: [72075186224037892][Partition][0][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-11-30T08:30:18.625078Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:85: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-11-30T08:30:18.625169Z node 2 :PERSQUEUE INFO: partition_init.cpp:1219: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7578432936017424585:2363]
2025-11-30T08:30:18.627167Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:85: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitFieldsStep
2025-11-30T08:30:18.627178Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:65: [rt3.dc1--topic1:0:Initializer] Initializing completed.
2025-11-30T08:30:18.627183Z node 2 :PERSQUEUE INFO: partition.cpp:705: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--topic1' partition 0 generation 1 [2:7578432936017424585:2363]
2025-11-30T08:30:18.627191Z node 2 :PERSQUEUE DEBUG: partition.cpp:725: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--topic1 partitition 0 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-11-30T08:30:18.627209Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:107: [72075186224037892][Partition][0][StateIdle] Initializing MLP Consumers: 0
2025-11-30T08:30:18.627912Z node 2 :PERSQUEUE DEBUG: partition.cpp:4399: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 1
2025-11-30T08:30:18.627924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.627934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2390: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TTransaction[ProposeConfig])
2025-11-30T08:30:18.628037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:18.628042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.628045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:18.628047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.628061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.628063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:18.628065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.628067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:18.628068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.628077Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:930: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-11-30T08:30:18.628085Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:70: [72075186224037892][Partition][0][StateIdle] Process MLP pending events. Count 0
2025-11-30T08:30:18.628416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3563: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-30T08:30:18.628463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:18.630198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-30T08:30:18.630447Z node 2 :PERSQUEUE DEBUG: partition.cpp:1418: [72075186224037892][Partition][0][StateIdle] Handle TEvPQ::TEvTxCommit Step 1764491418669, TxId 281474976720673
2025-11-30T08:30:18.630455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.630459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:18.630461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.630467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2454: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TTransaction[ProposeConfig])
2025-11-30T08:30:18.630488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:18.630491Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:18.630494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.630544Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:18.632529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:18.632725Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:107: [72075186224037892][Partition][0][StateIdle] Initializing MLP Consumers: 0
2025-11-30T08:30:18.632767Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:18.632774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.632778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.632779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.632781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.632783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.632848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:18.632964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:398: [PQ: 72075186224037892] Apply new config PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--topic1" Version: 0 LocalDC: true Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } TopicPath: "/Root/PQ/rt3.dc1--topic1" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Version: 0 Generation: 0 } MonitoringProjectId: ""
2025-11-30T08:30:18.632984Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:913: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-11-30T08:30:18.633038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3563: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-30T08:30:18.635222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-30T08:30:18.635248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3563: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-30T08:30:18.641550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
===Query complete
TClient::Ls request: /Root/PQ/rt3.dc1--topic1
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--topic1"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720673
    CreateStep: 1764491418669
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--topic1"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 600000...
(TRUNCATED)
GetTopicVersionFromPath:  record Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--topic1"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720673
    CreateStep: 1764491418669
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--topic1"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      YdbDatabasePath: "/Root"
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
    Partitions {
      PartitionId: 0
      TabletId: 72075186224037892
      Status: Active
    }
    AlterVersion: 1
    BalancerTabletID: 72075186224037893
    NextPartitionId: 1
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 1
      PlanResolution: 7
      Coordinators: 72057594046316545
      TimeCastBucketsPerMediator: 2
      Mediators: 72057594046382081
    }
    DomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    PathsInside: 12
    PathsLimit: 10000
    ShardsInside: 6
    ShardsLimit: 200000
    ResourcesDomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    DiskSpaceUsage {
      Tables {
        TotalSize: 0
        DataSize: 0
        IndexSize: 0
      }
      Topics {
        ReserveSize: 0
        AccountSize: 0
        DataSize: 0
        UsedReserveSize: 0
      }
    }
    PQPartitionsInside: 1
    PQPartitionsLimit: 1000000
    SchemeLimits {
      MaxDepth: 32
      MaxPaths: 10000
      MaxChildrenInDir: 100000
      MaxAclBytesSize: 10240
      MaxTableColumns: 200
      MaxTableColumnNameLength: 255
      MaxTableKeyColumns: 30
      MaxTableIndices: 20
      MaxShards: 200000
      MaxShardsInPath: 35000
      MaxConsistentCopyTargets: 10000
      MaxPathElementLength: 255
      ExtraPathSymbolsAllowed: "!\"#$%&\'()*+,-.:;<=>?@[\\]^_`{|}~"
      MaxPQPartitions: 1000000
      MaxTableCdcStreams: 5
      MaxExports: 10
      MaxImports: 10
      MaxColumnTableColumns: 10000
    }
  }
}
Path: "/Root/PQ/rt3.dc1--topic1"
 name rt3.dc1--topic1 version1
CallPersQueueGRPC request to localhost:12447
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--topic1"
  }
}
2025-11-30T08:30:18.653641Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
MetaResponse {
  CmdGetTopicMetadataResult {
    TopicInfo {
      Topic: "rt3.dc1--topic1"
      NumPartitions: 1
      Config {
        PartitionConfig {
          LifetimeSeconds: 86400
          LowWatermark: 8388608
          SourceIdLifetimeSeconds: 86400
          WriteSpeedInBytesPerSecond: 20000000
          BurstSize: 20000000
          SourceIdMaxCounts: 6000000
        }
        Version: 1
        LocalDC: true
        Codecs {
          Ids: 0
          Ids: 1
          Ids: 2
          Codecs: "raw"
          Codecs: "gzip"
          Codecs: "lzop"
        }
        TopicPath: "/Root/PQ/rt3.dc1--topic1"
        YdbDatabasePath: "/Root"
        Consumers {
          Name: "user"
          ReadFromTimestampsMs: 0
          FormatVersion: 0
          Version: 0
        }
      }
      ErrorCode: OK
    }
  }
}
=== Topic created, have version: 1
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:18.655665Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.656666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756132:3369], now have 1 active actors on pipe
2025-11-30T08:30:18.656889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.656900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.656933Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|58150806-bf19009e-7d794148-253e51e0_0 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:18.656947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.656952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.656959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:18.656961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.656967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.656978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:18.656981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:18.656985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.657023Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:18.657043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|58150806-bf19009e-7d794148-253e51e0_0"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:12447
2025-11-30T08:30:18.657197Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:18.661110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756132:3369] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|58150806-bf19009e-7d794148-253e51e0_0"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 0
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:18.670066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756135:3370], now have 1 active actors on pipe
2025-11-30T08:30:18.669419Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.672528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.672550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.672720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 0 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:18.673318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 0 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:18.673334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 0 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:18.673339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 0 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:18.673365Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:18.673390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.673395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.673408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.673410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.673414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:18.673418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.673424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.673462Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 0
2025-11-30T08:30:18.673490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.673496Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 1
2025-11-30T08:30:18.673499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.673502Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 2
2025-11-30T08:30:18.674595Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 0 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1048790 count 1 nextOffset 1 batches 3
2025-11-30T08:30:18.674613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:18.674616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:18.674620Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.674700Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 0,1 HeadOffset 0 endOffset 0 curOffset 1 d0000000000_00000000000000000000_00000_0000000001_00002? size 1048780 WTime 1764491418673
2025-11-30T08:30:18.675564Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:18.675577Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 1 size 1048780
2025-11-30T08:30:18.694513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 0 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:18.694541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:18.694569Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:18.694577Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.694587Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 0, partNo: 0, Offset: 0 is stored on disk
2025-11-30T08:30:18.694591Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.694597Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 0, partNo: 1, Offset: 0 is stored on disk
2025-11-30T08:30:18.694603Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.694606Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 0, partNo: 2, Offset: 0 is stored on disk
2025-11-30T08:30:18.694653Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:930: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-11-30T08:30:18.694662Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:972: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-11-30T08:30:18.694668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.694673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.694676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.694679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.694681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.694693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:18.694714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.694749Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--topic1' partition 0 user user offset 0 partno 0 count 1 size 1024000 endOffset 1 max time lag 0ms effective offset 0
2025-11-30T08:30:18.694762Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 1048780 count 1 last offset 0, current partition end offset: 1
2025-11-30T08:30:18.694764Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-11-30T08:30:18.694774Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:18.000000Z
2025-11-30T08:30:18.694778Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 3. All 1 blobs are from cache.
2025-11-30T08:30:18.694791Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 1 blobs
2025-11-30T08:30:18.694966Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 1 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:18.695104Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 1 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:18.695137Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 1 count 1 size 24704 from pos 0 cbcount 1
2025-11-30T08:30:18.695172Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1022: Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp done, result 1764491418673 queuesize 0 startOffset 0
2025-11-30T08:30:18.695404Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:18.695416Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:18.695985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756135:3370] destroyed
2025-11-30T08:30:18.695515Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 0
    Offset: 0
    PartNo: 0
    WriteTimestampMS: 1764491418673
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 19
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:18.700883Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.701574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756141:3373], now have 1 active actors on pipe
2025-11-30T08:30:18.701852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.701866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.701907Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|71b4e2f5-dbc3f222-aad27e09-eea78750_1 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:18.701921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.701925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.701932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:18.701935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.701939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.701952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:18.701955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:18.701958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.701993Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:18.702014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.702602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756141:3373] destroyed
2025-11-30T08:30:18.702211Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|71b4e2f5-dbc3f222-aad27e09-eea78750_1"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|71b4e2f5-dbc3f222-aad27e09-eea78750_1"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 1
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:18.717649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756144:3374], now have 1 active actors on pipe
2025-11-30T08:30:18.717050Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.718985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.719012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.719178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 1 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:18.719279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 1 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:18.719288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 1 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:18.719296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 1 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:18.719318Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:18.719337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.719341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.719349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.719353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.719358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:18.719360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.719364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.719397Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 0
2025-11-30T08:30:18.719410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.719413Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 1
2025-11-30T08:30:18.719416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.719419Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 2
2025-11-30T08:30:18.719689Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 1 partNo 2 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 1048790 count 1 nextOffset 2 batches 3
2025-11-30T08:30:18.719716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:18.719719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:18.719723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.719798Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 1,1 HeadOffset 1 endOffset 1 curOffset 2 d0000000000_00000000000000000001_00000_0000000001_00002? size 1048780 WTime 1764491418719
2025-11-30T08:30:18.720678Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:18.720705Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 1 partNo 0 count 1 size 1048780
2025-11-30T08:30:18.724067Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 1 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:18.724095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:18.724119Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:18.724118Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:18.724126Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.724137Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 1, partNo: 0, Offset: 1 is stored on disk
2025-11-30T08:30:18.724141Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.724147Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 1, partNo: 1, Offset: 1 is stored on disk
2025-11-30T08:30:18.724152Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.724157Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 1, partNo: 2, Offset: 1 is stored on disk
2025-11-30T08:30:18.724193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.724202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.724204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.724207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.724209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.724217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:18.724235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.724308Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:18.724885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756144:3374] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 1
    Offset: 1
    PartNo: 0
    WriteTimestampMS: 1764491418718
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:18.725592Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.726106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756152:3377], now have 1 active actors on pipe
2025-11-30T08:30:18.726124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.726129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.726159Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|19110bbc-860a8d60-1120320e-f2bf78b3_2 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:18.726168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.726171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.726175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:18.726177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.726180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.726188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:18.726191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:18.726194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.726206Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:18.726218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.726323Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:18.726628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756152:3377] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|19110bbc-860a8d60-1120320e-f2bf78b3_2"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:12447
2025-11-30T08:30:18.729177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.729201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.729204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.729212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.729214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|19110bbc-860a8d60-1120320e-f2bf78b3_2"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 2
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:18.734052Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.734603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756155:3378], now have 1 active actors on pipe
2025-11-30T08:30:18.735876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.735895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.736059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 2 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:18.736466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 2 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:18.736479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 2 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:18.736485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 2 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:18.736504Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:18.736522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.736525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.736535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.736538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.736543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:18.736545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.736549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.736581Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 0
2025-11-30T08:30:18.736593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.736597Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 1
2025-11-30T08:30:18.736602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.736605Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 2
2025-11-30T08:30:18.737015Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 2 partNo 2 FormedBlobsCount 0 NewHead: Offset 2 PartNo 0 PackedSize 1048790 count 1 nextOffset 3 batches 3
2025-11-30T08:30:18.737033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:18.737036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:18.737040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.737114Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 2,1 HeadOffset 2 endOffset 2 curOffset 3 d0000000000_00000000000000000002_00000_0000000001_00002? size 1048780 WTime 1764491418736
2025-11-30T08:30:18.737286Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:18.737306Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 2 partNo 0 count 1 size 1048780
2025-11-30T08:30:18.741214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 2 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:18.741240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:18.741264Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:18.741267Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:18.741271Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.741283Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 2, partNo: 0, Offset: 2 is stored on disk
2025-11-30T08:30:18.741287Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.741292Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 2, partNo: 1, Offset: 2 is stored on disk
2025-11-30T08:30:18.741297Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.741301Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 2, partNo: 2, Offset: 2 is stored on disk
2025-11-30T08:30:18.741342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.741347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.741349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.741351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.741353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.741363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:18.741380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.741489Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:18.741923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756155:3378] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 2
    Offset: 2
    PartNo: 0
    WriteTimestampMS: 1764491418736
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:18.743161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756160:3381], now have 1 active actors on pipe
2025-11-30T08:30:18.742720Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.743307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.743314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.743344Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b423ea22-2377d50e-fae8552e-253bd2f2_3 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:18.743352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.743354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.743357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:18.743359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.743361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.743369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:18.743371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:18.743373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.743385Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:18.743394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.743504Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:18.743797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756160:3381] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b423ea22-2377d50e-fae8552e-253bd2f2_3"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b423ea22-2377d50e-fae8552e-253bd2f2_3"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 3
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:18.749382Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.749840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756163:3382], now have 1 active actors on pipe
2025-11-30T08:30:18.751060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.751076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.751149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 3 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:18.751195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 3 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:18.751204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 3 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:18.751209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 3 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:18.751227Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:18.751245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.751248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.751257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.751260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.751263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:18.751265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.751270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.751301Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 0
2025-11-30T08:30:18.751317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.751320Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 1
2025-11-30T08:30:18.751323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.751326Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 2
2025-11-30T08:30:18.751526Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 3 partNo 2 FormedBlobsCount 0 NewHead: Offset 3 PartNo 0 PackedSize 1048790 count 1 nextOffset 4 batches 3
2025-11-30T08:30:18.751532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:18.751534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:18.751536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.751581Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 3,1 HeadOffset 3 endOffset 3 curOffset 4 d0000000000_00000000000000000003_00000_0000000001_00002? size 1048780 WTime 1764491418751
2025-11-30T08:30:18.751923Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:18.751933Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 3 partNo 0 count 1 size 1048780
2025-11-30T08:30:18.763227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 3 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:18.763263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:18.763302Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:18.763312Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.763330Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 3, partNo: 0, Offset: 3 is stored on disk
2025-11-30T08:30:18.763336Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.763342Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 3, partNo: 1, Offset: 3 is stored on disk
2025-11-30T08:30:18.763351Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.763357Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 3, partNo: 2, Offset: 3 is stored on disk
2025-11-30T08:30:18.763421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.763426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.763428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.763433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.763436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.763449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:18.763474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.763673Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:18.763788Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:18.764564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756163:3382] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 3
    Offset: 3
    PartNo: 0
    WriteTimestampMS: 1764491418751
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 11
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:18.765952Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.768441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756168:3385], now have 1 active actors on pipe
2025-11-30T08:30:18.768601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.768614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.768659Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|39478658-1a09fbd-4fd41373-347c218a_4 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:18.768674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.768678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.768685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:18.768688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.768693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.768708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:18.768711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:18.768717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.768738Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:18.768759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.768959Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|39478658-1a09fbd-4fd41373-347c218a_4"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|39478658-1a09fbd-4fd41373-347c218a_4"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 4
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:18.775444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756168:3385] destroyed
2025-11-30T08:30:18.783777Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.784573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756172:3387], now have 1 active actors on pipe
2025-11-30T08:30:18.786062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.786083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.786151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 4 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:18.786254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 4 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:18.786263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 4 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:18.786268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 4 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:18.786290Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:18.786310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.786314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.786322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.786329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.786334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:18.786336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.786343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.786374Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 0
2025-11-30T08:30:18.786388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.786391Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 1
2025-11-30T08:30:18.786394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.786397Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 2
2025-11-30T08:30:18.787178Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 4 partNo 2 FormedBlobsCount 0 NewHead: Offset 4 PartNo 0 PackedSize 1048790 count 1 nextOffset 5 batches 3
2025-11-30T08:30:18.787193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:18.787197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:18.787201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.787278Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 4,1 HeadOffset 4 endOffset 4 curOffset 5 d0000000000_00000000000000000004_00000_0000000001_00002? size 1048780 WTime 1764491418787
2025-11-30T08:30:18.787620Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:18.787635Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 4 partNo 0 count 1 size 1048780
2025-11-30T08:30:18.791955Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 4 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:18.791979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:18.792003Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:18.792010Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.792020Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 4, partNo: 0, Offset: 4 is stored on disk
2025-11-30T08:30:18.792025Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.792030Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 4, partNo: 1, Offset: 4 is stored on disk
2025-11-30T08:30:18.792035Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.792042Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 4, partNo: 2, Offset: 4 is stored on disk
2025-11-30T08:30:18.792081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.792085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.792087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.792090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.792092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.792101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:18.792120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.792257Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 2 suffix '63' size 1048780
CallPersQueueGRPC response:
2025-11-30T08:30:18.792809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756172:3387] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 4
    Offset: 4
    PartNo: 0
    WriteTimestampMS: 1764491418784
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 6
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:18.792349Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:18.793652Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.794053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756177:3390], now have 1 active actors on pipe
2025-11-30T08:30:18.794228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.794239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.794267Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|23f13e14-c669c9fa-6a704457-b236e8d7_5 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:18.794278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.794281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.794285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:18.794287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.794290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.794297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:18.794301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:18.794303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.794322Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:18.794333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.794380Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-30T08:30:18.794702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756177:3390] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|23f13e14-c669c9fa-6a704457-b236e8d7_5"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|23f13e14-c669c9fa-6a704457-b236e8d7_5"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 5
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:18.802918Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.803433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756181:3391], now have 1 active actors on pipe
2025-11-30T08:30:18.804684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.804703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.804775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 5 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:18.804906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 5 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:18.804916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 5 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:18.804922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 5 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:18.804944Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:18.804962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.804967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.804976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.804980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.804984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:18.804989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.804993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.805026Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 0
2025-11-30T08:30:18.805038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.805042Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 1
2025-11-30T08:30:18.805045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.805049Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 2
2025-11-30T08:30:18.805626Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 5 partNo 2 FormedBlobsCount 0 NewHead: Offset 5 PartNo 0 PackedSize 1048790 count 1 nextOffset 6 batches 3
2025-11-30T08:30:18.805640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:18.805643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:18.805647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.805715Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 5,1 HeadOffset 5 endOffset 5 curOffset 6 d0000000000_00000000000000000005_00000_0000000001_00002? size 1048780 WTime 1764491418805
2025-11-30T08:30:18.805913Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:18.805926Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 5 partNo 0 count 1 size 1048780
2025-11-30T08:30:18.810545Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 5 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:18.810568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:18.810595Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:18.810604Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.810615Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 5, partNo: 0, Offset: 5 is stored on disk
2025-11-30T08:30:18.810619Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.810624Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 5, partNo: 1, Offset: 5 is stored on disk
2025-11-30T08:30:18.810631Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.810635Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 5, partNo: 2, Offset: 5 is stored on disk
2025-11-30T08:30:18.810675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.810678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.810681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.810683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.810686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.810695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:18.810713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.810854Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:18.810946Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:18.811368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756181:3391] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 5
    Offset: 5
    PartNo: 0
    WriteTimestampMS: 1764491418804
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:18.813684Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.814691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756186:3395], now have 1 active actors on pipe
2025-11-30T08:30:18.814713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.814719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.814750Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b113801e-fd0e8935-c634b830-57ec3c57_6 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:18.814761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.814765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.814771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:18.814773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.814777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.814789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:18.814792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:18.814796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.814812Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:18.814827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.814995Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b113801e-fd0e8935-c634b830-57ec3c57_6"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:12447
2025-11-30T08:30:18.815533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756186:3395] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b113801e-fd0e8935-c634b830-57ec3c57_6"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 6
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:18.820770Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.822750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756190:3397], now have 1 active actors on pipe
2025-11-30T08:30:18.824140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.824162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.824231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 6 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:18.824284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 6 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:18.824315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 6 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:18.824321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 6 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:18.824344Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:18.824363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.824368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.824378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.824381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.824388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:18.824390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.824394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.824427Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 0
2025-11-30T08:30:18.824441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.824443Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 1
2025-11-30T08:30:18.824446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.824449Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 2
2025-11-30T08:30:18.825078Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 6 partNo 2 FormedBlobsCount 0 NewHead: Offset 6 PartNo 0 PackedSize 1048790 count 1 nextOffset 7 batches 3
2025-11-30T08:30:18.825100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:18.825103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:18.825106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.825181Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 6,1 HeadOffset 6 endOffset 6 curOffset 7 d0000000000_00000000000000000006_00000_0000000001_00002? size 1048780 WTime 1764491418824
2025-11-30T08:30:18.825387Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:18.825400Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 6 partNo 0 count 1 size 1048780
2025-11-30T08:30:18.828880Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 6 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:18.828909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:18.828934Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:18.828940Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.828956Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 6, partNo: 0, Offset: 6 is stored on disk
2025-11-30T08:30:18.828964Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.828970Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 6, partNo: 1, Offset: 6 is stored on disk
2025-11-30T08:30:18.828975Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.828980Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 6, partNo: 2, Offset: 6 is stored on disk
2025-11-30T08:30:18.829017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.829030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.829032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.829035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.829037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.829046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:18.829063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.829077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.829078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.829080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.829081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.829082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.829133Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:18.829206Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 6
    Offset: 6
    PartNo: 0
    WriteTimestampMS: 1764491418824
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-30T08:30:18.829662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756190:3397] destroyed
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:18.830750Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.831479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756195:3400], now have 1 active actors on pipe
2025-11-30T08:30:18.831499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.831507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.831537Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|3e8ad6d8-2390c116-a5efcafe-29015a34_7 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:18.831548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.831551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.831556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:18.831557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.831561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.831571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:18.831573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:18.831577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.831591Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:18.831610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.831821Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|3e8ad6d8-2390c116-a5efcafe-29015a34_7"
    Status: Active
    SeqNo: 0
  }
}
2025-11-30T08:30:18.832140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756195:3400] destroyed
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|3e8ad6d8-2390c116-a5efcafe-29015a34_7"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 7
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:18.838839Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.839460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756198:3401], now have 1 active actors on pipe
2025-11-30T08:30:18.840491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.840515Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.840601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 7 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:18.840664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 7 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:18.840680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 7 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:18.840687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 7 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:18.840704Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:18.840722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.840730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.840739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.840741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.840744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:18.840746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.840749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.840803Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 0
2025-11-30T08:30:18.840827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.840831Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 1
2025-11-30T08:30:18.840834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.840837Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 2
2025-11-30T08:30:18.841373Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 7 partNo 2 FormedBlobsCount 0 NewHead: Offset 7 PartNo 0 PackedSize 1048790 count 1 nextOffset 8 batches 3
2025-11-30T08:30:18.841391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:18.841395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:18.841398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.841476Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 7,1 HeadOffset 7 endOffset 7 curOffset 8 d0000000000_00000000000000000007_00000_0000000001_00002? size 1048780 WTime 1764491418841
2025-11-30T08:30:18.841666Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:18.841686Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 7 partNo 0 count 1 size 1048780
2025-11-30T08:30:18.844679Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 7 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:18.844702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:18.844728Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:18.844738Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.844750Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 7, partNo: 0, Offset: 7 is stored on disk
2025-11-30T08:30:18.844758Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.844764Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 7, partNo: 1, Offset: 7 is stored on disk
2025-11-30T08:30:18.844769Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.844774Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 7, partNo: 2, Offset: 7 is stored on disk
2025-11-30T08:30:18.844826Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:18.844833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.844837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.844839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.844842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.844844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.844856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000000_00000_0000000001_00002? (1048780)
2025-11-30T08:30:18.844860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000001_00000_0000000001_00002? (1048780)
2025-11-30T08:30:18.844863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000002_00000_0000000001_00002? (1048780)
2025-11-30T08:30:18.844867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000003_00000_0000000001_00002? (1048780)
2025-11-30T08:30:18.844870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000004_00000_0000000001_00002? (1048780)
2025-11-30T08:30:18.844872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000005_00000_0000000001_00002? (1048780)
2025-11-30T08:30:18.844875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000006_00000_0000000001_00002? (1048780)
2025-11-30T08:30:18.844877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000007_00000_0000000001_00002? (1048780)
2025-11-30T08:30:18.844881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 8 keys were taken away. Let's read 8390240 bytes
2025-11-30T08:30:18.844902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.844917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 8 blobs
2025-11-30T08:30:18.844923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000000_00000_0000000001_00002?
2025-11-30T08:30:18.844933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000001_00000_0000000001_00002?
2025-11-30T08:30:18.844947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000002_00000_0000000001_00002?
2025-11-30T08:30:18.844967Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000003_00000_0000000001_00002?
2025-11-30T08:30:18.844978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000004_00000_0000000001_00002?
2025-11-30T08:30:18.845004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000005_00000_0000000001_00002?
2025-11-30T08:30:18.845017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000006_00000_0000000001_00002?
2025-11-30T08:30:18.845027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000007_00000_0000000001_00002?
2025-11-30T08:30:18.845041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 8 blobs for compaction
2025-11-30T08:30:18.845052Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 1 times before, last time 2025-11-30T08:30:18.000000Z
2025-11-30T08:30:18.845059Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 1 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:18.000000Z
2025-11-30T08:30:18.845063Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 2 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:18.000000Z
2025-11-30T08:30:18.845067Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 3 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:18.000000Z
2025-11-30T08:30:18.845071Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 4 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:18.000000Z
2025-11-30T08:30:18.845075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 5 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:18.000000Z
2025-11-30T08:30:18.845088Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 6 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:18.000000Z
2025-11-30T08:30:18.845094Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 7 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:18.000000Z
2025-11-30T08:30:18.845106Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 8 blobs are from cache.
2025-11-30T08:30:18.845134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-30T08:30:18.845142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/8] d0000000000_00000000000000000000_00000_0000000001_00002?
2025-11-30T08:30:18.845145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:18.845148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000000_00000_0000000001_00002?, parameters.CurOffset=0
2025-11-30T08:30:18.845284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 0.0/3
2025-11-30T08:30:18.845293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 0
2025-11-30T08:30:18.845300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 0:0 appended
2025-11-30T08:30:18.845439Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 0.1/3
2025-11-30T08:30:18.845442Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 1
2025-11-30T08:30:18.845445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 0:1 appended
2025-11-30T08:30:18.845458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 0.2/3
2025-11-30T08:30:18.845461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 2
2025-11-30T08:30:18.845699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 0 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1048790 count 1 nextOffset 1 batches 3
2025-11-30T08:30:18.845704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 0:2 appended
2025-11-30T08:30:18.845707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/8] d0000000000_00000000000000000001_00000_0000000001_00002?
2025-11-30T08:30:18.845708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:18.845710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000001_00000_0000000001_00002?, parameters.CurOffset=1
2025-11-30T08:30:18.845839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 1.0/3
2025-11-30T08:30:18.845842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 0
2025-11-30T08:30:18.845843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 1:0 appended
2025-11-30T08:30:18.846137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 1.1/3
2025-11-30T08:30:18.846140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 1
2025-11-30T08:30:18.846141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 1:1 appended
2025-11-30T08:30:18.846157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 1.2/3
2025-11-30T08:30:18.846160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 2
2025-11-30T08:30:18.846387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 1 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2097545 count 2 nextOffset 2 batches 5
2025-11-30T08:30:18.846393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 1:2 appended
2025-11-30T08:30:18.846398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/8] d0000000000_00000000000000000002_00000_0000000001_00002?
2025-11-30T08:30:18.846400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:18.846404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000002_00000_0000000001_00002?, parameters.CurOffset=2
2025-11-30T08:30:18.846529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 2.0/3
2025-11-30T08:30:18.846531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 0
2025-11-30T08:30:18.846533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 2:0 appended
2025-11-30T08:30:18.847090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 2.1/3
2025-11-30T08:30:18.847093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 1
2025-11-30T08:30:18.847094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 2:1 appended
2025-11-30T08:30:18.847104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 2.2/3
2025-11-30T08:30:18.847106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 2
2025-11-30T08:30:18.847504Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:18.847509Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:18.847512Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:18.847515Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:18.847519Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:18.847521Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:18.847524Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:18.847528Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:18.847712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 2 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3146300 count 3 nextOffset 3 batches 7
2025-11-30T08:30:18.847730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 2:2 appended
2025-11-30T08:30:18.847734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/8] d0000000000_00000000000000000003_00000_0000000001_00002?
2025-11-30T08:30:18.847737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:18.847740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000003_00000_0000000001_00002?, parameters.CurOffset=3
2025-11-30T08:30:18.847870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 3.0/3
2025-11-30T08:30:18.847873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 0
2025-11-30T08:30:18.847876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 3:0 appended
2025-11-30T08:30:18.847993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 3.1/3
2025-11-30T08:30:18.847995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 1
2025-11-30T08:30:18.847995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 3:1 appended
2025-11-30T08:30:18.848003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 3.2/3
2025-11-30T08:30:18.848005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 2
2025-11-30T08:30:18.848480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 3 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4195055 count 4 nextOffset 4 batches 9
2025-11-30T08:30:18.848483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 3:2 appended
2025-11-30T08:30:18.848485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/8] d0000000000_00000000000000000004_00000_0000000001_00002?
2025-11-30T08:30:18.848487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:18.848488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000004_00000_0000000001_00002?, parameters.CurOffset=4
2025-11-30T08:30:18.848625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 4.0/3
2025-11-30T08:30:18.848629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 0
2025-11-30T08:30:18.848632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 4:0 appended
2025-11-30T08:30:18.848877Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-30T08:30:18.849537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 4.1/3
2025-11-30T08:30:18.849549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 1
2025-11-30T08:30:18.849553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 4:1 appended
2025-11-30T08:30:18.849566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 4.2/3
2025-11-30T08:30:18.849569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 2
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 7
    Offset: 7
    PartNo: 0
    WriteTimestampMS: 1764491418840
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:18.849748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 4 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 5243810 count 5 nextOffset 5 batches 11
2025-11-30T08:30:18.849762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 4:2 appended
2025-11-30T08:30:18.849767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/8] d0000000000_00000000000000000005_00000_0000000001_00002?
2025-11-30T08:30:18.849768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:18.849771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000005_00000_0000000001_00002?, parameters.CurOffset=5
2025-11-30T08:30:18.849924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 5.0/3
2025-11-30T08:30:18.849934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 0
2025-11-30T08:30:18.849936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 5:0 appended
2025-11-30T08:30:18.850081Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 5.1/3
2025-11-30T08:30:18.850089Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 1
2025-11-30T08:30:18.850091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 5:1 appended
2025-11-30T08:30:18.850103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 5.2/3
2025-11-30T08:30:18.850108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 2
2025-11-30T08:30:18.850645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 5 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 6292565 count 6 nextOffset 6 batches 13
2025-11-30T08:30:18.850657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 5:2 appended
2025-11-30T08:30:18.850661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/8] d0000000000_00000000000000000006_00000_0000000001_00002?
2025-11-30T08:30:18.850663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:18.850665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000006_00000_0000000001_00002?, parameters.CurOffset=6
2025-11-30T08:30:18.850799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 6.0/3
2025-11-30T08:30:18.850808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 0
2025-11-30T08:30:18.850811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 6:0 appended
2025-11-30T08:30:18.850947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 6.1/3
2025-11-30T08:30:18.850961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 1
2025-11-30T08:30:18.850963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 6:1 appended
2025-11-30T08:30:18.850974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 6.2/3
2025-11-30T08:30:18.850976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 2
2025-11-30T08:30:18.851329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 6 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 7341320 count 7 nextOffset 7 batches 15
2025-11-30T08:30:18.851333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 6:2 appended
2025-11-30T08:30:18.851335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/8] d0000000000_00000000000000000007_00000_0000000001_00002?
2025-11-30T08:30:18.851337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:18.851339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000007_00000_0000000001_00002?, parameters.CurOffset=7
2025-11-30T08:30:18.851466Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 7.0/3
2025-11-30T08:30:18.851469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 0
2025-11-30T08:30:18.851471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 7:0 appended
2025-11-30T08:30:18.851924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 7.1/3
2025-11-30T08:30:18.851928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 1
2025-11-30T08:30:18.851929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 7:1 appended
2025-11-30T08:30:18.851940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 7.2/3
2025-11-30T08:30:18.851941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 2
2025-11-30T08:30:18.852710Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.859629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 7 partNo 2 result is x0000000000_00000000000000000000_00000_0000000007_00016 size 8365339
2025-11-30T08:30:18.859660Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1106: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000000_00000_0000000007_00016 new key d0000000000_00000000000000000000_00000_0000000007_00016 size 8365339 WTime 1764491418859
2025-11-30T08:30:18.859710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 7 partNo 2 FormedBlobsCount 1 NewHead: Offset 7 PartNo 2 PackedSize 24736 count 1 nextOffset 8 batches 1
2025-11-30T08:30:18.859720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 7:2 appended
2025-11-30T08:30:18.859778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:636: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 7,1 HeadOffset 0 endOffset 0 curOffset 8 d0000000000_00000000000000000007_00002_0000000001_00000| size 24726 WTime 1764491418859
2025-11-30T08:30:18.859840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756198:3401] destroyed
2025-11-30T08:30:18.859854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756203:3404], now have 1 active actors on pipe
2025-11-30T08:30:18.859883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.859894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.859916Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:18.859931Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 7 size 8365339
2025-11-30T08:30:18.859942Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 7 partNo 2 count 1 size 24726
2025-11-30T08:30:18.859980Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b7d0dc9a-54ae1ff0-29bb0d4c-957ec9e0_8 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:18.859995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.859998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.860005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:18.860007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.860010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.860021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:18.860024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:18.860027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.860045Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:18.860105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.860591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756203:3404] destroyed
2025-11-30T08:30:18.860197Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b7d0dc9a-54ae1ff0-29bb0d4c-957ec9e0_8"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b7d0dc9a-54ae1ff0-29bb0d4c-957ec9e0_8"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 8
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:18.868421Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.870463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756208:3406], now have 1 active actors on pipe
2025-11-30T08:30:18.871564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.871585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.871668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 8 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:18.871734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 8 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:18.871745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 8 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:18.871749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 8 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:18.871791Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:18.871812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.871816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.871828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.871831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.871836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:18.871839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.871843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.871880Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 0
2025-11-30T08:30:18.871895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.871899Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 1
2025-11-30T08:30:18.871902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.871906Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 2
2025-11-30T08:30:18.872129Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 8 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 1048790 count 1 nextOffset 9 batches 3
2025-11-30T08:30:18.872138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:18.872141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:18.872143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.872194Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 8,1 HeadOffset 8 endOffset 8 curOffset 9 d0000000000_00000000000000000008_00000_0000000001_00002? size 1048780 WTime 1764491418872
2025-11-30T08:30:18.872506Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:18.872517Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 8 partNo 0 count 1 size 1048780
2025-11-30T08:30:18.874846Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 0 count 7 size 8365339 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:18.874859Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 7 count 1 size 24726 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:18.874922Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 7 parts 16 suffix '0' size 8365339
2025-11-30T08:30:18.874941Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 7 partno 2 count 1 parts 0 suffix '124' size 24726
2025-11-30T08:30:18.875036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:18.875088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2279: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-30T08:30:18.875097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:18.881224Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 8 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:18.881261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:18.881292Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:18.881303Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.881316Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 8, partNo: 0, Offset: 8 is stored on disk
2025-11-30T08:30:18.881321Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.881325Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 8, partNo: 1, Offset: 8 is stored on disk
2025-11-30T08:30:18.881337Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.881342Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 8, partNo: 2, Offset: 8 is stored on disk
2025-11-30T08:30:18.881402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.881405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.881407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.881411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.881414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.881425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000000_00000_0000000001_00002?
2025-11-30T08:30:18.881430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000001_00000_0000000001_00002?
2025-11-30T08:30:18.881434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000002_00000_0000000001_00002?
2025-11-30T08:30:18.881437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000003_00000_0000000001_00002?
2025-11-30T08:30:18.881441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000004_00000_0000000001_00002?
2025-11-30T08:30:18.881444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000005_00000_0000000001_00002?
2025-11-30T08:30:18.881447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000006_00000_0000000001_00002?
2025-11-30T08:30:18.881450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000007_00000_0000000001_00002?
2025-11-30T08:30:18.881478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:18.881502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.881522Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:18.881533Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000000_00000_0000000001_00002?(+) to d0000000000_00000000000000000000_00000_0000000001_00002?(+)
2025-11-30T08:30:18.881535Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000001_00000_0000000001_00002?(+) to d0000000000_00000000000000000001_00000_0000000001_00002?(+)
2025-11-30T08:30:18.881537Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000002_00000_0000000001_00002?(+) to d0000000000_00000000000000000002_00000_0000000001_00002?(+)
2025-11-30T08:30:18.881539Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000003_00000_0000000001_00002?(+) to d0000000000_00000000000000000003_00000_0000000001_00002?(+)
2025-11-30T08:30:18.881556Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000004_00000_0000000001_00002?(+) to d0000000000_00000000000000000004_00000_0000000001_00002?(+)
2025-11-30T08:30:18.881558Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000005_00000_0000000001_00002?(+) to d0000000000_00000000000000000005_00000_0000000001_00002?(+)
2025-11-30T08:30:18.881561Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000006_00000_0000000001_00002?(+) to d0000000000_00000000000000000006_00000_0000000001_00002?(+)
2025-11-30T08:30:18.881564Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000007_00000_0000000001_00002?(+) to d0000000000_00000000000000000007_00000_0000000001_00002?(+)
2025-11-30T08:30:18.881990Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:18.882130Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:18.882536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756208:3406] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 8
    Offset: 8
    PartNo: 0
    WriteTimestampMS: 1764491418871
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 9
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:18.883294Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 0 count 1 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:18.883307Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 1 count 1 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:18.883313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 2 count 1 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:18.883317Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 3 count 1 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:18.883322Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 4 count 1 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:18.883326Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 5 count 1 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:18.883330Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 6 count 1 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:18.883333Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 7 count 1 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:18.883350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:18.883324Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.883367Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:18.883381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.883389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.883391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.883394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.883398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.883401Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:18.883406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:18.883414Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:18.883428Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:18.883438Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:18.883444Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:18.883451Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:18.883456Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:18.883462Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:18.883772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756215:3411], now have 1 active actors on pipe
2025-11-30T08:30:18.883877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.883888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.884047Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:18.883924Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|911f492-984d39dc-b4370c35-9fb46762_9 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:18.883938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.883942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.883946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:18.883948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.883951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.883961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:18.883964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:18.883968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.883980Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:18.883993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.884289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756215:3411] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|911f492-984d39dc-b4370c35-9fb46762_9"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|911f492-984d39dc-b4370c35-9fb46762_9"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 9
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:18.892990Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.897689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756219:3413], now have 1 active actors on pipe
2025-11-30T08:30:18.898389Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.898407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.898475Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 9 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:18.898533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 9 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:18.898547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 9 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:18.898551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 9 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:18.898571Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:18.898595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.898604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.898612Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.898614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.898618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:18.898621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.898625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.898656Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 0
2025-11-30T08:30:18.898674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.898682Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 1
2025-11-30T08:30:18.898686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.898689Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 2
2025-11-30T08:30:18.898913Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 9 partNo 2 FormedBlobsCount 0 NewHead: Offset 9 PartNo 0 PackedSize 1048790 count 1 nextOffset 10 batches 3
2025-11-30T08:30:18.898931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:18.898935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:18.898938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.899000Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 9,1 HeadOffset 9 endOffset 9 curOffset 10 d0000000000_00000000000000000009_00000_0000000001_00002? size 1048780 WTime 1764491418898
2025-11-30T08:30:18.899144Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:18.899166Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 9 partNo 0 count 1 size 1048780
2025-11-30T08:30:18.907521Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-30T08:30:18.907207Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 9 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:18.907231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 9
    Offset: 9
    PartNo: 0
    WriteTimestampMS: 1764491418898
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 8
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-30T08:30:18.907256Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:18.907263Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.907274Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 9, partNo: 0, Offset: 9 is stored on disk
2025-11-30T08:30:18.907278Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.907283Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 9, partNo: 1, Offset: 9 is stored on disk
2025-11-30T08:30:18.907289Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.907294Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 9, partNo: 2, Offset: 9 is stored on disk
2025-11-30T08:30:18.907330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.907333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.907336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.907338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.907342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.907352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:18.907367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.907419Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 2 suffix '63' size 1048780
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:18.908003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756219:3413] destroyed
2025-11-30T08:30:18.909009Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.909552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756224:3416], now have 1 active actors on pipe
2025-11-30T08:30:18.909674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.909683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.909714Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|13aff0a7-beb1f0ef-30ca1295-131ad7d6_10 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:18.909723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.909726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.909730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:18.909732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.909735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.909743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:18.909745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:18.909747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.909761Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:18.909771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.909877Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|13aff0a7-beb1f0ef-30ca1295-131ad7d6_10"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:12447
2025-11-30T08:30:18.910412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756224:3416] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|13aff0a7-beb1f0ef-30ca1295-131ad7d6_10"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 10
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:18.917196Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.917804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756227:3417], now have 1 active actors on pipe
2025-11-30T08:30:18.918874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.918898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.918974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 10 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:18.919042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 10 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:18.919056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 10 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:18.919065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 10 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:18.919085Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:18.919109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.919116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.919125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.919128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.919132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:18.919135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.919139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.919170Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 0
2025-11-30T08:30:18.919189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.919196Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 1
2025-11-30T08:30:18.919200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.919202Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 2
2025-11-30T08:30:18.919507Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 10 partNo 2 FormedBlobsCount 0 NewHead: Offset 10 PartNo 0 PackedSize 1048790 count 1 nextOffset 11 batches 3
2025-11-30T08:30:18.919533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:18.919535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:18.919539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.919610Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 10,1 HeadOffset 10 endOffset 10 curOffset 11 d0000000000_00000000000000000010_00000_0000000001_00002? size 1048780 WTime 1764491418919
2025-11-30T08:30:18.919825Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:18.919845Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 10 partNo 0 count 1 size 1048780
2025-11-30T08:30:18.922841Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 10 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:18.922862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:18.922886Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:18.922887Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:18.922893Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.922902Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 10, partNo: 0, Offset: 10 is stored on disk
2025-11-30T08:30:18.922907Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.922911Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 10, partNo: 1, Offset: 10 is stored on disk
2025-11-30T08:30:18.922916Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.922920Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 10, partNo: 2, Offset: 10 is stored on disk
2025-11-30T08:30:18.922951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.922953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.922955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.922958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.922961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.922968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:18.922983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.923076Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 10
    Offset: 10
    PartNo: 0
    WriteTimestampMS: 1764491418918
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-30T08:30:18.923520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756227:3417] destroyed
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:18.924249Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.924635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756232:3420], now have 1 active actors on pipe
2025-11-30T08:30:18.924814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.924822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.924852Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|3624c80d-4e91a29-442384ff-22bb783a_11 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:18.924860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.924863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.924867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:18.924869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.924872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.924879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:18.924882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:18.924885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.924897Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:18.924908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.924966Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|3624c80d-4e91a29-442384ff-22bb783a_11"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:12447
2025-11-30T08:30:18.925332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756232:3420] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|3624c80d-4e91a29-442384ff-22bb783a_11"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 11
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:18.929709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.929723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.929725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.929727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.929729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.936416Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.937077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756236:3422], now have 1 active actors on pipe
2025-11-30T08:30:18.938353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.938375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.938446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 11 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:18.938498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 11 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:18.938511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 11 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:18.938517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 11 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:18.938542Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:18.938566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.938573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.938582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.938588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.938596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:18.938598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.938602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.938633Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 0
2025-11-30T08:30:18.938664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.938675Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 1
2025-11-30T08:30:18.938680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.938683Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 2
2025-11-30T08:30:18.938932Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 11 partNo 2 FormedBlobsCount 0 NewHead: Offset 11 PartNo 0 PackedSize 1048790 count 1 nextOffset 12 batches 3
2025-11-30T08:30:18.938945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:18.938949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:18.938951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.939025Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 11,1 HeadOffset 11 endOffset 11 curOffset 12 d0000000000_00000000000000000011_00000_0000000001_00002? size 1048780 WTime 1764491418938
2025-11-30T08:30:18.939196Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:18.939209Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 11 partNo 0 count 1 size 1048780
2025-11-30T08:30:18.945868Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 11 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:18.945898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:18.945922Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:18.945924Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:18.945930Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.945941Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 11, partNo: 0, Offset: 11 is stored on disk
2025-11-30T08:30:18.945946Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.945954Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 11, partNo: 1, Offset: 11 is stored on disk
2025-11-30T08:30:18.945959Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.945964Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 11, partNo: 2, Offset: 11 is stored on disk
2025-11-30T08:30:18.946006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.946013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.946015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.946018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.946020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.946027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:18.946044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.946111Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 11
    Offset: 11
    PartNo: 0
    WriteTimestampMS: 1764491418938
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 7
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:18.946628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756236:3422] destroyed
2025-11-30T08:30:18.947196Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.947796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756241:3425], now have 1 active actors on pipe
2025-11-30T08:30:18.947872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.947882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.947916Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|2c2081e5-416cb833-1eb71439-a046ee72_12 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:18.947929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.947933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.947938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:18.947942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.947945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.947953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:18.947956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:18.947959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.947972Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:18.947984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.948036Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:18.948263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756241:3425] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|2c2081e5-416cb833-1eb71439-a046ee72_12"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|2c2081e5-416cb833-1eb71439-a046ee72_12"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 12
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:18.955549Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.956137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756247:3429], now have 1 active actors on pipe
2025-11-30T08:30:18.957456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.957473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.957536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 12 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:18.957590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 12 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:18.957603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 12 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:18.957607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 12 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:18.957625Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:18.957648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.957655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.957663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.957667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.957675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:18.957677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.957681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.957710Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 0
2025-11-30T08:30:18.957725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.957732Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 1
2025-11-30T08:30:18.957735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.957737Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 2
2025-11-30T08:30:18.957982Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 12 partNo 2 FormedBlobsCount 0 NewHead: Offset 12 PartNo 0 PackedSize 1048790 count 1 nextOffset 13 batches 3
2025-11-30T08:30:18.958010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:18.958017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:18.958020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.958083Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 12,1 HeadOffset 12 endOffset 12 curOffset 13 d0000000000_00000000000000000012_00000_0000000001_00002? size 1048780 WTime 1764491418958
2025-11-30T08:30:18.958437Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:18.958457Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 12 partNo 0 count 1 size 1048780
2025-11-30T08:30:18.962322Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 12 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:18.962347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:18.962372Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:18.962373Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:18.962383Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.962393Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 12, partNo: 0, Offset: 12 is stored on disk
2025-11-30T08:30:18.962399Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.962402Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 12, partNo: 1, Offset: 12 is stored on disk
2025-11-30T08:30:18.962407Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.962410Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 12, partNo: 2, Offset: 12 is stored on disk
2025-11-30T08:30:18.962443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.962449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.962451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.962453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.962455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.962462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:18.962477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.962553Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:18.962885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756247:3429] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 12
    Offset: 12
    PartNo: 0
    WriteTimestampMS: 1764491418957
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:18.963783Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.964538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756252:3432], now have 1 active actors on pipe
2025-11-30T08:30:18.964562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.964568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.964599Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b4a2f96d-a965dbd8-2c726ecd-533bd5b3_13 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:18.964612Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.964615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.964621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:18.964622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.964625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.964633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:18.964635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:18.964637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.964651Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:18.964663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b4a2f96d-a965dbd8-2c726ecd-533bd5b3_13"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:12447
2025-11-30T08:30:18.964734Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:18.966407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756252:3432] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b4a2f96d-a965dbd8-2c726ecd-533bd5b3_13"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 13
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:18.971296Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.971967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756255:3433], now have 1 active actors on pipe
2025-11-30T08:30:18.974272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.974293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.974364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 13 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:18.974433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 13 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:18.974441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 13 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:18.974445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 13 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:18.974468Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:18.974487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.974490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.974501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.974504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.974508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:18.974509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.974513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.974544Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 0
2025-11-30T08:30:18.974557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.974560Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 1
2025-11-30T08:30:18.974563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.974565Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 2
2025-11-30T08:30:18.974866Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 13 partNo 2 FormedBlobsCount 0 NewHead: Offset 13 PartNo 0 PackedSize 1048790 count 1 nextOffset 14 batches 3
2025-11-30T08:30:18.974875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:18.974877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:18.974880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.974962Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 13,1 HeadOffset 13 endOffset 13 curOffset 14 d0000000000_00000000000000000013_00000_0000000001_00002? size 1048780 WTime 1764491418971
2025-11-30T08:30:18.975831Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:18.975843Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 13 partNo 0 count 1 size 1048780
2025-11-30T08:30:18.980576Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 13 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:18.980600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:18.980626Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:18.980634Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.980646Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 13, partNo: 0, Offset: 13 is stored on disk
2025-11-30T08:30:18.980650Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.980654Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 13, partNo: 1, Offset: 13 is stored on disk
2025-11-30T08:30:18.980661Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.980665Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 13, partNo: 2, Offset: 13 is stored on disk
2025-11-30T08:30:18.980699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.980702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.980703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.980705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.980707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.980715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:18.980730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.980785Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:18.980877Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 13
    Offset: 13
    PartNo: 0
    WriteTimestampMS: 1764491418971
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:18.982257Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.982954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756255:3433] destroyed
2025-11-30T08:30:18.982970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756260:3436], now have 1 active actors on pipe
2025-11-30T08:30:18.982981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.982987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.983016Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|e06e90cc-534316b1-e7b97a58-79292627_14 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:18.983033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.983037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.983042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:18.983048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.983051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.983059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:18.983061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:18.983063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.983170Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|e06e90cc-534316b1-e7b97a58-79292627_14"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:12447
2025-11-30T08:30:18.983076Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:18.983090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.983570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756260:3436] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|e06e90cc-534316b1-e7b97a58-79292627_14"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 14
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:18.990639Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.991278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756264:3437], now have 1 active actors on pipe
2025-11-30T08:30:18.992550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.992569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.992639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 14 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:18.992704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 14 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:18.992717Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 14 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:18.992722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 14 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:18.992744Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:18.992766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.992775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.992783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.992829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.992833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:18.992834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.992838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.992868Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 0
2025-11-30T08:30:18.992879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.992882Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 1
2025-11-30T08:30:18.992884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.992887Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 2
2025-11-30T08:30:18.993169Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 14 partNo 2 FormedBlobsCount 0 NewHead: Offset 14 PartNo 0 PackedSize 1048790 count 1 nextOffset 15 batches 3
2025-11-30T08:30:18.993186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:18.993188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:18.993191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.993256Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 14,1 HeadOffset 14 endOffset 14 curOffset 15 d0000000000_00000000000000000014_00000_0000000001_00002? size 1048780 WTime 1764491418992
2025-11-30T08:30:18.993413Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:18.993430Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 14 partNo 0 count 1 size 1048780
2025-11-30T08:30:18.996787Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 14 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:18.996823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:18.996870Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:18.997025Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:492: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-11-30T08:30:18.997048Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:18.997054Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.997064Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 14, partNo: 0, Offset: 14 is stored on disk
2025-11-30T08:30:18.997068Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.997071Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 14, partNo: 1, Offset: 14 is stored on disk
2025-11-30T08:30:18.997077Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:18.997080Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 14, partNo: 2, Offset: 14 is stored on disk
2025-11-30T08:30:18.997109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.997112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.997114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.997117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:18.997118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.997126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:18.997141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.997280Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 14
    Offset: 14
    PartNo: 0
    WriteTimestampMS: 1764491418986
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 10
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:18.998489Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:18.999098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756264:3437] destroyed
2025-11-30T08:30:18.999111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432937684756269:3441], now have 1 active actors on pipe
2025-11-30T08:30:18.999120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:18.999126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:18.999151Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|fe803d6a-cd022eb2-86453b60-1bbb65c7_15 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:18.999160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:18.999163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:18.999168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:18.999170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:18.999188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:18.999197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:18.999200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:18.999202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:18.999215Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:18.999227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:18.999366Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|fe803d6a-cd022eb2-86453b60-1bbb65c7_15"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|fe803d6a-cd022eb2-86453b60-1bbb65c7_15"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 15
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:19.002810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432937684756269:3441] destroyed
2025-11-30T08:30:19.006012Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.006521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723568:3442], now have 1 active actors on pipe
2025-11-30T08:30:19.007372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.007384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.007478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 15 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:19.007525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 15 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:19.007531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 15 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:19.007535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 15 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:19.007549Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:19.007563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.007566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.007572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.007573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.007576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:19.007577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.007580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.007604Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 0
2025-11-30T08:30:19.007615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.007617Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 1
2025-11-30T08:30:19.007619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.007621Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 2
2025-11-30T08:30:19.007934Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 15 partNo 2 FormedBlobsCount 0 NewHead: Offset 15 PartNo 0 PackedSize 1048790 count 1 nextOffset 16 batches 3
2025-11-30T08:30:19.007943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:19.007946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:19.007949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.008017Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 15,1 HeadOffset 15 endOffset 15 curOffset 16 d0000000000_00000000000000000015_00000_0000000001_00002? size 1048780 WTime 1764491419005
2025-11-30T08:30:19.008178Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:19.008194Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 15 partNo 0 count 1 size 1048780
2025-11-30T08:30:19.011627Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 15 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.011646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:19.011668Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:19.011674Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.011685Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 15, partNo: 0, Offset: 15 is stored on disk
2025-11-30T08:30:19.011689Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.011694Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 15, partNo: 1, Offset: 15 is stored on disk
2025-11-30T08:30:19.011713Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.011718Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 15, partNo: 2, Offset: 15 is stored on disk
2025-11-30T08:30:19.011750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.011753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.011755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.011757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.011759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.011769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000008_00000_0000000001_00002? (1048780)
2025-11-30T08:30:19.011772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000009_00000_0000000001_00002? (1048780)
2025-11-30T08:30:19.011775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000010_00000_0000000001_00002? (1048780)
2025-11-30T08:30:19.011778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000011_00000_0000000001_00002? (1048780)
2025-11-30T08:30:19.011780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000012_00000_0000000001_00002? (1048780)
2025-11-30T08:30:19.011782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000013_00000_0000000001_00002? (1048780)
2025-11-30T08:30:19.011784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000014_00000_0000000001_00002? (1048780)
2025-11-30T08:30:19.011787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000015_00000_0000000001_00002? (1048780)
2025-11-30T08:30:19.011790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 8 keys were taken away. Let's read 8390240 bytes
2025-11-30T08:30:19.011806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.011818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 8 blobs
2025-11-30T08:30:19.011821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000008_00000_0000000001_00002?
2025-11-30T08:30:19.011830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000009_00000_0000000001_00002?
2025-11-30T08:30:19.011837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000010_00000_0000000001_00002?
2025-11-30T08:30:19.011848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000011_00000_0000000001_00002?
2025-11-30T08:30:19.011857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000012_00000_0000000001_00002?
2025-11-30T08:30:19.011871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000013_00000_0000000001_00002?
2025-11-30T08:30:19.011880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000014_00000_0000000001_00002?
2025-11-30T08:30:19.011887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000015_00000_0000000001_00002?
2025-11-30T08:30:19.011896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 8 blobs for compaction
2025-11-30T08:30:19.011905Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 8 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:18.000000Z
2025-11-30T08:30:19.011908Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 9 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:18.000000Z
2025-11-30T08:30:19.011912Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 10 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:18.000000Z
2025-11-30T08:30:19.011915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 11 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:18.000000Z
2025-11-30T08:30:19.011918Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 12 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:18.000000Z
2025-11-30T08:30:19.011921Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 13 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:18.000000Z
2025-11-30T08:30:19.011923Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 14 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:18.000000Z
2025-11-30T08:30:19.011926Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 15 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:19.000000Z
2025-11-30T08:30:19.011933Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 8 blobs are from cache.
2025-11-30T08:30:19.011960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-30T08:30:19.011968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/8] d0000000000_00000000000000000008_00000_0000000001_00002?
2025-11-30T08:30:19.011970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:19.011973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000008_00000_0000000001_00002?, parameters.CurOffset=8
2025-11-30T08:30:19.012116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 8.0/3
2025-11-30T08:30:19.012125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 0
2025-11-30T08:30:19.012130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 8:0 appended
2025-11-30T08:30:19.012282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 8.1/3
2025-11-30T08:30:19.012286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 1
2025-11-30T08:30:19.012288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 8:1 appended
2025-11-30T08:30:19.012300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 8.2/3
2025-11-30T08:30:19.012303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 2
2025-11-30T08:30:19.012536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 8 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 1048790 count 1 nextOffset 9 batches 3
2025-11-30T08:30:19.012542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 8:2 appended
2025-11-30T08:30:19.012547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/8] d0000000000_00000000000000000009_00000_0000000001_00002?
2025-11-30T08:30:19.012549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:19.012552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000009_00000_0000000001_00002?, parameters.CurOffset=9
2025-11-30T08:30:19.012685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 9.0/3
2025-11-30T08:30:19.012689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 0
2025-11-30T08:30:19.012691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 9:0 appended
2025-11-30T08:30:19.013136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 9.1/3
2025-11-30T08:30:19.013140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 1
2025-11-30T08:30:19.013141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 9:1 appended
2025-11-30T08:30:19.013158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 9.2/3
2025-11-30T08:30:19.013161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 2
2025-11-30T08:30:19.013250Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.013265Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:19.013268Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:19.013272Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:19.013275Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:19.013278Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:19.013282Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:19.013286Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:19.013290Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:19.013321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 9 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 2097545 count 2 nextOffset 10 batches 5
2025-11-30T08:30:19.013325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 9:2 appended
2025-11-30T08:30:19.013327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/8] d0000000000_00000000000000000010_00000_0000000001_00002?
2025-11-30T08:30:19.013328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:19.013329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000010_00000_0000000001_00002?, parameters.CurOffset=10
2025-11-30T08:30:19.013435Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:19.013470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 10.0/3
2025-11-30T08:30:19.013480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 0
2025-11-30T08:30:19.013481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 10:0 appended
2025-11-30T08:30:19.013828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 10.1/3
2025-11-30T08:30:19.013832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 1
2025-11-30T08:30:19.013835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 10:1 appended
2025-11-30T08:30:19.013850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 10.2/3
2025-11-30T08:30:19.013852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 2
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 15
    Offset: 15
    PartNo: 0
    WriteTimestampMS: 1764491419005
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:19.014018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 10 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 3146300 count 3 nextOffset 11 batches 7
2025-11-30T08:30:19.014030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 10:2 appended
2025-11-30T08:30:19.014032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/8] d0000000000_00000000000000000011_00000_0000000001_00002?
2025-11-30T08:30:19.014033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:19.014035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000011_00000_0000000001_00002?, parameters.CurOffset=11
2025-11-30T08:30:19.014173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 11.0/3
2025-11-30T08:30:19.014175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 0
2025-11-30T08:30:19.014176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 11:0 appended
2025-11-30T08:30:19.014311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 11.1/3
2025-11-30T08:30:19.014312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 1
2025-11-30T08:30:19.014313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 11:1 appended
2025-11-30T08:30:19.014323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 11.2/3
2025-11-30T08:30:19.014324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 2
2025-11-30T08:30:19.014475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 11 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 4195055 count 4 nextOffset 12 batches 9
2025-11-30T08:30:19.014477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 11:2 appended
2025-11-30T08:30:19.014479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/8] d0000000000_00000000000000000012_00000_0000000001_00002?
2025-11-30T08:30:19.014480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:19.014481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000012_00000_0000000001_00002?, parameters.CurOffset=12
2025-11-30T08:30:19.014872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 12.0/3
2025-11-30T08:30:19.014875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 0
2025-11-30T08:30:19.014876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 12:0 appended
2025-11-30T08:30:19.014994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 12.1/3
2025-11-30T08:30:19.014995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 1
2025-11-30T08:30:19.014997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 12:1 appended
2025-11-30T08:30:19.015003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 12.2/3
2025-11-30T08:30:19.015004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 2
2025-11-30T08:30:19.015153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 12 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 5243810 count 5 nextOffset 13 batches 11
2025-11-30T08:30:19.015155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 12:2 appended
2025-11-30T08:30:19.015157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/8] d0000000000_00000000000000000013_00000_0000000001_00002?
2025-11-30T08:30:19.015159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:19.015161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000013_00000_0000000001_00002?, parameters.CurOffset=13
2025-11-30T08:30:19.015291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 13.0/3
2025-11-30T08:30:19.015293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 0
2025-11-30T08:30:19.015294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 13:0 appended
2025-11-30T08:30:19.015414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 13.1/3
2025-11-30T08:30:19.015416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 1
2025-11-30T08:30:19.015417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 13:1 appended
2025-11-30T08:30:19.015424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 13.2/3
2025-11-30T08:30:19.015426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 2
2025-11-30T08:30:19.015585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 13 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 6292565 count 6 nextOffset 14 batches 13
2025-11-30T08:30:19.015588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 13:2 appended
2025-11-30T08:30:19.015590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/8] d0000000000_00000000000000000014_00000_0000000001_00002?
2025-11-30T08:30:19.015591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:19.015593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000014_00000_0000000001_00002?, parameters.CurOffset=14
2025-11-30T08:30:19.015747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 14.0/3
2025-11-30T08:30:19.015751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 0
2025-11-30T08:30:19.015752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 14:0 appended
2025-11-30T08:30:19.016061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 14.1/3
2025-11-30T08:30:19.016065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 1
2025-11-30T08:30:19.016067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 14:1 appended
2025-11-30T08:30:19.016080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 14.2/3
2025-11-30T08:30:19.016083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 2
2025-11-30T08:30:19.016276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 14 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 7341320 count 7 nextOffset 15 batches 15
2025-11-30T08:30:19.016280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 14:2 appended
2025-11-30T08:30:19.016284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/8] d0000000000_00000000000000000015_00000_0000000001_00002?
2025-11-30T08:30:19.016286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:19.016289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000015_00000_0000000001_00002?, parameters.CurOffset=15
2025-11-30T08:30:19.016431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 15.0/3
2025-11-30T08:30:19.016436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 0
2025-11-30T08:30:19.016438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 15:0 appended
2025-11-30T08:30:19.016581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 15.1/3
2025-11-30T08:30:19.016584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 1
2025-11-30T08:30:19.017782Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.021582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 15 partNo 1 result is x0000000000_00000000000000000007_00002_0000000008_00015 size 7878063
2025-11-30T08:30:19.021602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 15:1 appended
2025-11-30T08:30:19.021625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 15.2/3
2025-11-30T08:30:19.021629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 2
2025-11-30T08:30:19.021641Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1106: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000007_00002_0000000008_00015 new key d0000000000_00000000000000000007_00002_0000000008_00015 size 7878063 WTime 1764491419020
2025-11-30T08:30:19.021780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 15 partNo 2 FormedBlobsCount 1 NewHead: Offset 15 PartNo 1 PackedSize 536763 count 1 nextOffset 16 batches 2
2025-11-30T08:30:19.021786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 15:2 appended
2025-11-30T08:30:19.021839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:636: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 15,1 HeadOffset 7 endOffset 8 curOffset 16 d0000000000_00000000000000000015_00001_0000000001_00001| size 536753 WTime 1764491419020
2025-11-30T08:30:19.021992Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723568:3442] destroyed
2025-11-30T08:30:19.022000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723573:3445], now have 1 active actors on pipe
2025-11-30T08:30:19.022023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.022029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.022043Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:19.022056Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 7 partNo 2 count 8 size 7878063
2025-11-30T08:30:19.022062Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 15 partNo 1 count 1 size 536753
2025-11-30T08:30:19.022098Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|6db59004-5723c99e-87b1a64f-6e4aa6ae_16 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:19.022107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.022110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.022116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:19.022119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.022122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.022130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:19.022133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:19.022325Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:19.022135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.022151Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:19.022198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.022689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723573:3445] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|6db59004-5723c99e-87b1a64f-6e4aa6ae_16"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|6db59004-5723c99e-87b1a64f-6e4aa6ae_16"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 16
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:19.030357Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.030382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.030399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.030402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.030405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.030407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.030962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723577:3447], now have 1 active actors on pipe
2025-11-30T08:30:19.032103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.032125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.032200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 16 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:19.032281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 16 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:19.032297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 16 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:19.032307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 16 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:19.032324Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:19.032346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.032354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.032362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.032369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.032373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:19.032376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.032381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.032411Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 0
2025-11-30T08:30:19.032427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.032434Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 1
2025-11-30T08:30:19.032438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.032445Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 2
2025-11-30T08:30:19.032750Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 16 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 1048790 count 1 nextOffset 17 batches 3
2025-11-30T08:30:19.032770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:19.032775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:19.032778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.032870Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 16,1 HeadOffset 16 endOffset 16 curOffset 17 d0000000000_00000000000000000016_00000_0000000001_00002? size 1048780 WTime 1764491419032
2025-11-30T08:30:19.033039Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:19.033054Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 16 partNo 0 count 1 size 1048780
2025-11-30T08:30:19.038836Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 16 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.038862Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 7 count 8 size 7878063 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.038866Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 15 count 1 size 536753 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.038874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:19.038898Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:19.038904Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.038905Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.038915Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 16, partNo: 0, Offset: 16 is stored on disk
2025-11-30T08:30:19.038916Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 7 partno 2 count 8 parts 15 suffix '0' size 7878063
2025-11-30T08:30:19.038920Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 15 partno 1 count 1 parts 1 suffix '124' size 536753
2025-11-30T08:30:19.038920Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.038924Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 16, partNo: 1, Offset: 16 is stored on disk
2025-11-30T08:30:19.038931Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.038936Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 16, partNo: 2, Offset: 16 is stored on disk
2025-11-30T08:30:19.038972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.038976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.038979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.038981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.038984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.038991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:178: [72075186224037892][Partition][0][StateIdle] Blobs compaction in progress
2025-11-30T08:30:19.038996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:19.039095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.039096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.039098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.039100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.039102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.039107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000007_00002_0000000001_00000|
2025-11-30T08:30:19.039110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000008_00000_0000000001_00002?
2025-11-30T08:30:19.039113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000009_00000_0000000001_00002?
2025-11-30T08:30:19.039115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000010_00000_0000000001_00002?
2025-11-30T08:30:19.039120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000011_00000_0000000001_00002?
2025-11-30T08:30:19.039122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000012_00000_0000000001_00002?
2025-11-30T08:30:19.039125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000013_00000_0000000001_00002?
2025-11-30T08:30:19.039127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000014_00000_0000000001_00002?
2025-11-30T08:30:19.039129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000015_00000_0000000001_00002?
2025-11-30T08:30:19.039147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:19.039165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.039184Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:19.039192Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000007_00002_0000000001_00000|(+) to d0000000000_00000000000000000007_00002_0000000001_00000|(+)
2025-11-30T08:30:19.039194Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000008_00000_0000000001_00002?(+) to d0000000000_00000000000000000008_00000_0000000001_00002?(+)
2025-11-30T08:30:19.039196Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000009_00000_0000000001_00002?(+) to d0000000000_00000000000000000009_00000_0000000001_00002?(+)
2025-11-30T08:30:19.039198Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000010_00000_0000000001_00002?(+) to d0000000000_00000000000000000010_00000_0000000001_00002?(+)
2025-11-30T08:30:19.039200Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000011_00000_0000000001_00002?(+) to d0000000000_00000000000000000011_00000_0000000001_00002?(+)
2025-11-30T08:30:19.039202Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000012_00000_0000000001_00002?(+) to d0000000000_00000000000000000012_00000_0000000001_00002?(+)
2025-11-30T08:30:19.039204Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000013_00000_0000000001_00002?(+) to d0000000000_00000000000000000013_00000_0000000001_00002?(+)
2025-11-30T08:30:19.039207Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000014_00000_0000000001_00002?(+) to d0000000000_00000000000000000014_00000_0000000001_00002?(+)
2025-11-30T08:30:19.039209Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000015_00000_0000000001_00002?(+) to d0000000000_00000000000000000015_00000_0000000001_00002?(+)
2025-11-30T08:30:19.039917Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 16
    Offset: 16
    PartNo: 0
    WriteTimestampMS: 1764491419032
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:19.041056Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.040421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723577:3447] destroyed
2025-11-30T08:30:19.040510Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 7 count 1 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.040515Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 8 count 1 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.040521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 9 count 1 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.040525Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 10 count 1 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.040532Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 11 count 1 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.040536Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 12 count 1 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.040540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 13 count 1 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.040543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 14 count 1 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.040549Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 15 count 1 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.040560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:19.040574Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:19.040581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.040583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.040585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.040585Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 7 partno 2 count 1 parts 0 suffix '124' size 24726
2025-11-30T08:30:19.040587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.040589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.040592Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.040594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:19.040601Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.040609Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.040614Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.040619Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.040623Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.040628Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.040633Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.041743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723585:3452], now have 1 active actors on pipe
2025-11-30T08:30:19.041918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.041930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.041972Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|7d9fad42-764cdce9-b7e279e7-a00dc8bb_17 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:19.041983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.041986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.041991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:19.041993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.041996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.042005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:19.042007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:19.042009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.042021Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:19.042043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.042082Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:19.042305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723585:3452] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|7d9fad42-764cdce9-b7e279e7-a00dc8bb_17"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|7d9fad42-764cdce9-b7e279e7-a00dc8bb_17"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 17
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:19.048660Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.052287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723589:3454], now have 1 active actors on pipe
2025-11-30T08:30:19.052682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.052697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.052776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 17 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:19.052863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 17 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:19.052881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 17 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:19.052887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 17 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:19.052914Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:19.052938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.052948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.052957Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.052964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.052968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:19.052970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.052974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.053006Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 0
2025-11-30T08:30:19.053022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.053026Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 1
2025-11-30T08:30:19.053029Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.053031Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 2
2025-11-30T08:30:19.053242Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 17 partNo 2 FormedBlobsCount 0 NewHead: Offset 17 PartNo 0 PackedSize 1048790 count 1 nextOffset 18 batches 3
2025-11-30T08:30:19.053252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:19.053254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:19.053256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.053305Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 17,1 HeadOffset 17 endOffset 17 curOffset 18 d0000000000_00000000000000000017_00000_0000000001_00002? size 1048780 WTime 1764491419052
2025-11-30T08:30:19.053438Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:19.053458Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 17 partNo 0 count 1 size 1048780
2025-11-30T08:30:19.056521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 17 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.056543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:19.056560Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.056566Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:19.056572Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.056584Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 17, partNo: 0, Offset: 17 is stored on disk
2025-11-30T08:30:19.056592Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.056597Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 17, partNo: 1, Offset: 17 is stored on disk
2025-11-30T08:30:19.056602Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.056606Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 17, partNo: 2, Offset: 17 is stored on disk
2025-11-30T08:30:19.056639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.056646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.056647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.056650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.056652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.056660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:19.056675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.056747Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:19.057098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723589:3454] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 17
    Offset: 17
    PartNo: 0
    WriteTimestampMS: 1764491419052
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:19.058053Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.058374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723594:3457], now have 1 active actors on pipe
2025-11-30T08:30:19.058490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.058496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.058520Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|5d4c0bdd-20e0ec63-4c2266f7-4f776cf4_18 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:19.058526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.058541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.058545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:19.058547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.058550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.058558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:19.058561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:19.058563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.058574Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:19.058584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.058641Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:19.058822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723594:3457] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|5d4c0bdd-20e0ec63-4c2266f7-4f776cf4_18"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|5d4c0bdd-20e0ec63-4c2266f7-4f776cf4_18"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 18
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:19.065473Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.066162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723597:3458], now have 1 active actors on pipe
2025-11-30T08:30:19.067452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.067470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.067561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 18 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:19.067625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 18 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:19.067636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 18 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:19.067642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 18 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:19.067662Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:19.067682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.067687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.067696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.067714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.067720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:19.067722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.067727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.067758Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 0
2025-11-30T08:30:19.067772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.067775Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 1
2025-11-30T08:30:19.067779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.067782Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 2
2025-11-30T08:30:19.068068Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 18 partNo 2 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 1048790 count 1 nextOffset 19 batches 3
2025-11-30T08:30:19.068077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:19.068080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:19.068083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.068154Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 18,1 HeadOffset 18 endOffset 18 curOffset 19 d0000000000_00000000000000000018_00000_0000000001_00002? size 1048780 WTime 1764491419068
2025-11-30T08:30:19.068309Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:19.068322Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 18 partNo 0 count 1 size 1048780
2025-11-30T08:30:19.074903Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 18 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.074927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:19.074952Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:19.074961Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.074974Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 18, partNo: 0, Offset: 18 is stored on disk
2025-11-30T08:30:19.074979Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.074984Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 18, partNo: 1, Offset: 18 is stored on disk
2025-11-30T08:30:19.074990Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.074994Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 18, partNo: 2, Offset: 18 is stored on disk
2025-11-30T08:30:19.075033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.075037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.075039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.075043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.075045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.075055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:19.075073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.075219Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.075332Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 18
    Offset: 18
    PartNo: 0
    WriteTimestampMS: 1764491419067
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 7
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:19.076657Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.076828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723597:3458] destroyed
2025-11-30T08:30:19.077640Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9f67b354-654fdcf5-8ddbe812-598bae54_19"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9f67b354-654fdcf5-8ddbe812-598bae54_19"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 19
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:19.077356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723602:3461], now have 1 active actors on pipe
2025-11-30T08:30:19.077384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.077390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.077423Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9f67b354-654fdcf5-8ddbe812-598bae54_19 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:19.077434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.077438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.077443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:19.077444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.077448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.077458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:19.077461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:19.077464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.077478Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:19.077491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.078069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723602:3461] destroyed
2025-11-30T08:30:19.085983Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.086713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723605:3462], now have 1 active actors on pipe
2025-11-30T08:30:19.087967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.087977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.088047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 19 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:19.088098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 19 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:19.088111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 19 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:19.088115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 19 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:19.088134Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:19.088177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.088183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.088192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.088195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.088199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:19.088201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.088205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.088237Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 0
2025-11-30T08:30:19.088251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.088254Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 1
2025-11-30T08:30:19.088258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.088260Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 2
2025-11-30T08:30:19.088558Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 19 partNo 2 FormedBlobsCount 0 NewHead: Offset 19 PartNo 0 PackedSize 1048790 count 1 nextOffset 20 batches 3
2025-11-30T08:30:19.088570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:19.088573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:19.088576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.088654Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 19,1 HeadOffset 19 endOffset 19 curOffset 20 d0000000000_00000000000000000019_00000_0000000001_00002? size 1048780 WTime 1764491419087
2025-11-30T08:30:19.088913Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:19.088935Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 19 partNo 0 count 1 size 1048780
2025-11-30T08:30:19.103107Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 19 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.103148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:19.103170Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.103177Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:19.103184Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.103200Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 19, partNo: 0, Offset: 19 is stored on disk
2025-11-30T08:30:19.103206Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.103212Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 19, partNo: 1, Offset: 19 is stored on disk
2025-11-30T08:30:19.103217Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.103222Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 19, partNo: 2, Offset: 19 is stored on disk
2025-11-30T08:30:19.103264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.103267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.103269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.103271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.103273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.103283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:19.103301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.103408Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:19.103829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723605:3462] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 19
    Offset: 19
    PartNo: 0
    WriteTimestampMS: 1764491419087
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 15
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:19.105164Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.105627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723613:3467], now have 1 active actors on pipe
2025-11-30T08:30:19.108965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.108983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.109025Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|2f3942be-5bf69071-932fd30-6442ebd1_20 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:19.109038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.109045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.109050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:19.109052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.109055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.109067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:19.109070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:19.109073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.109091Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:19.109119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.112897Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|2f3942be-5bf69071-932fd30-6442ebd1_20"
    Status: Active
    SeqNo: 0
  }
}
2025-11-30T08:30:19.113300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723613:3467] destroyed
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|2f3942be-5bf69071-932fd30-6442ebd1_20"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 20
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:19.120767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723616:3468], now have 1 active actors on pipe
2025-11-30T08:30:19.120221Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.122057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.122073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.122146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 20 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:19.122199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 20 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:19.122210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 20 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:19.122215Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 20 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:19.122236Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:19.122257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.122260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.122269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.122272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.122276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:19.122278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.122282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.122311Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 0
2025-11-30T08:30:19.122323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.122326Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 1
2025-11-30T08:30:19.122329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.122332Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 2
2025-11-30T08:30:19.122657Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 20 partNo 2 FormedBlobsCount 0 NewHead: Offset 20 PartNo 0 PackedSize 1048790 count 1 nextOffset 21 batches 3
2025-11-30T08:30:19.122666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:19.122669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:19.122671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.122741Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 20,1 HeadOffset 20 endOffset 20 curOffset 21 d0000000000_00000000000000000020_00000_0000000001_00002? size 1048780 WTime 1764491419121
2025-11-30T08:30:19.122899Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:19.122917Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 20 partNo 0 count 1 size 1048780
2025-11-30T08:30:19.133541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2279: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-30T08:30:19.142691Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 20 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.142729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:19.142760Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:19.142769Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.142782Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 20, partNo: 0, Offset: 20 is stored on disk
2025-11-30T08:30:19.142794Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.142799Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 20, partNo: 1, Offset: 20 is stored on disk
2025-11-30T08:30:19.142808Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.142813Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 20, partNo: 2, Offset: 20 is stored on disk
2025-11-30T08:30:19.142865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.142872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.142874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.142877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.142879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.142888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:19.142907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.142969Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.144973Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 20
    Offset: 20
    PartNo: 0
    WriteTimestampMS: 1764491419121
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 19
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:19.146094Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.148836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723616:3468] destroyed
2025-11-30T08:30:19.148858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723622:3472], now have 1 active actors on pipe
2025-11-30T08:30:19.151147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.151190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.151252Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|24f0ed6e-4b9476a0-e0fa582c-595036f_21 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:19.151275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.151285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.151293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:19.151295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.151300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.151316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:19.151324Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:19.151328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.151350Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:19.151379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.151481Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-30T08:30:19.151912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723622:3472] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|24f0ed6e-4b9476a0-e0fa582c-595036f_21"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|24f0ed6e-4b9476a0-e0fa582c-595036f_21"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 21
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:19.157166Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.158222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723626:3473], now have 1 active actors on pipe
2025-11-30T08:30:19.159609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.159632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.159738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 21 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:19.159795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 21 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:19.159810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 21 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:19.159816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 21 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:19.159839Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:19.159862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.159870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.159882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.159885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.159890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:19.159904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.159910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.159950Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 0
2025-11-30T08:30:19.159967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.159970Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 1
2025-11-30T08:30:19.159974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.159977Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 2
2025-11-30T08:30:19.160206Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 21 partNo 2 FormedBlobsCount 0 NewHead: Offset 21 PartNo 0 PackedSize 1048790 count 1 nextOffset 22 batches 3
2025-11-30T08:30:19.160217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:19.160221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:19.160224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.160305Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 21,1 HeadOffset 21 endOffset 21 curOffset 22 d0000000000_00000000000000000021_00000_0000000001_00002? size 1048780 WTime 1764491419159
2025-11-30T08:30:19.160463Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:19.160477Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 21 partNo 0 count 1 size 1048780
2025-11-30T08:30:19.165093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 21 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.165124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:19.165154Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:19.165157Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.165163Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.165175Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 21, partNo: 0, Offset: 21 is stored on disk
2025-11-30T08:30:19.165181Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.165185Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 21, partNo: 1, Offset: 21 is stored on disk
2025-11-30T08:30:19.165192Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.165200Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 21, partNo: 2, Offset: 21 is stored on disk
2025-11-30T08:30:19.165247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.165251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.165254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.165257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.165260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.165271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:19.165291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.165404Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 21
    Offset: 21
    PartNo: 0
    WriteTimestampMS: 1764491419159
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:19.166267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723626:3473] destroyed
2025-11-30T08:30:19.169220Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.172787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723631:3477], now have 1 active actors on pipe
2025-11-30T08:30:19.172860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.172871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.172930Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|cf928651-3078d96b-e864ae9e-7a19fd4b_22 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:19.172945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.172950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.172960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:19.172963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.172968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.172985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:19.172989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:19.172994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.173031Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:19.173053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.173153Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|cf928651-3078d96b-e864ae9e-7a19fd4b_22"
    Status: Active
    SeqNo: 0
  }
}
2025-11-30T08:30:19.173923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723631:3477] destroyed
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|cf928651-3078d96b-e864ae9e-7a19fd4b_22"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 22
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:19.180051Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.180534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723634:3478], now have 1 active actors on pipe
2025-11-30T08:30:19.181615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.181638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.181715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 22 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:19.181782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 22 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:19.181794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 22 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:19.181800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 22 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:19.181819Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:19.181837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.181842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.181850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.181853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.181857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:19.181863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.181867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.181896Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 0
2025-11-30T08:30:19.181908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.181911Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 1
2025-11-30T08:30:19.181914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.181917Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 2
2025-11-30T08:30:19.182357Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 22 partNo 2 FormedBlobsCount 0 NewHead: Offset 22 PartNo 0 PackedSize 1048790 count 1 nextOffset 23 batches 3
2025-11-30T08:30:19.182369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:19.182372Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:19.182376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.182448Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 22,1 HeadOffset 22 endOffset 22 curOffset 23 d0000000000_00000000000000000022_00000_0000000001_00002? size 1048780 WTime 1764491419182
2025-11-30T08:30:19.182626Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:19.182641Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 22 partNo 0 count 1 size 1048780
2025-11-30T08:30:19.185567Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 22 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.185592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:19.185613Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.185619Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:19.185625Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.185634Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 22, partNo: 0, Offset: 22 is stored on disk
2025-11-30T08:30:19.185643Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.185647Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 22, partNo: 1, Offset: 22 is stored on disk
2025-11-30T08:30:19.185653Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.185657Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 22, partNo: 2, Offset: 22 is stored on disk
2025-11-30T08:30:19.185690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.185697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.185701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.185704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.185706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.185715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:19.185732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.185807Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-30T08:30:19.186174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723634:3478] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 22
    Offset: 22
    PartNo: 0
    WriteTimestampMS: 1764491419181
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:19.186826Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.187207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723639:3481], now have 1 active actors on pipe
2025-11-30T08:30:19.187291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.187304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.187334Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|bb892d60-77466dcf-9a96d807-a9f38700_23 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:19.187347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.187350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.187354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:19.187361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.187365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.187373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:19.187380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:19.187383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.187395Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:19.187405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.187529Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-30T08:30:19.187771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723639:3481] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|bb892d60-77466dcf-9a96d807-a9f38700_23"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|bb892d60-77466dcf-9a96d807-a9f38700_23"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 23
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:19.192216Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.195160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723642:3482], now have 1 active actors on pipe
2025-11-30T08:30:19.195686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.195726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.195824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 23 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:19.195884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 23 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:19.195901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 23 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:19.195908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 23 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:19.195932Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:19.195962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.195973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.195986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.195994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.196000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:19.196003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.196008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.196051Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 0
2025-11-30T08:30:19.196072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.196084Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 1
2025-11-30T08:30:19.196088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.196092Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 2
2025-11-30T08:30:19.196351Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 23 partNo 2 FormedBlobsCount 0 NewHead: Offset 23 PartNo 0 PackedSize 1048790 count 1 nextOffset 24 batches 3
2025-11-30T08:30:19.196374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:19.196378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:19.196382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.196469Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 23,1 HeadOffset 23 endOffset 23 curOffset 24 d0000000000_00000000000000000023_00000_0000000001_00002? size 1048780 WTime 1764491419196
2025-11-30T08:30:19.196632Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:19.196646Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 23 partNo 0 count 1 size 1048780
2025-11-30T08:30:19.207279Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 23 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.207314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:19.207348Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:19.207358Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.207373Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 23, partNo: 0, Offset: 23 is stored on disk
2025-11-30T08:30:19.207380Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.207385Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 23, partNo: 1, Offset: 23 is stored on disk
2025-11-30T08:30:19.207393Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.207398Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 23, partNo: 2, Offset: 23 is stored on disk
2025-11-30T08:30:19.207451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.207456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.207459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.207463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.207465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.207482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000016_00000_0000000001_00002? (1048780)
2025-11-30T08:30:19.207489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000017_00000_0000000001_00002? (1048780)
2025-11-30T08:30:19.207495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000018_00000_0000000001_00002? (1048780)
2025-11-30T08:30:19.207499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000019_00000_0000000001_00002? (1048780)
2025-11-30T08:30:19.207502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00000_0000000001_00002? (1048780)
2025-11-30T08:30:19.207505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00002? (1048780)
2025-11-30T08:30:19.207508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000001_00002? (1048780)
2025-11-30T08:30:19.207511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000023_00000_0000000001_00002? (1048780)
2025-11-30T08:30:19.207514Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 8 keys were taken away. Let's read 8390240 bytes
2025-11-30T08:30:19.207539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.207560Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 8 blobs
2025-11-30T08:30:19.207564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000016_00000_0000000001_00002?
2025-11-30T08:30:19.207576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000017_00000_0000000001_00002?
2025-11-30T08:30:19.207588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000018_00000_0000000001_00002?
2025-11-30T08:30:19.207591Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.207602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000019_00000_0000000001_00002?
2025-11-30T08:30:19.207614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00000_0000000001_00002?
2025-11-30T08:30:19.207635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00002?
2025-11-30T08:30:19.207645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000001_00002?
2025-11-30T08:30:19.207662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000023_00000_0000000001_00002?
2025-11-30T08:30:19.207672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 8 blobs for compaction
2025-11-30T08:30:19.207684Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:19.207697Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 16 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:19.000000Z
2025-11-30T08:30:19.207715Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 17 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:19.000000Z
2025-11-30T08:30:19.207720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 18 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:19.000000Z
2025-11-30T08:30:19.207723Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 19 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:19.000000Z
2025-11-30T08:30:19.207728Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 20 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:19.000000Z
2025-11-30T08:30:19.207732Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 21 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:19.000000Z
2025-11-30T08:30:19.207736Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 22 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:19.000000Z
2025-11-30T08:30:19.207740Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 23 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:19.000000Z
2025-11-30T08:30:19.207755Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 8 blobs are from cache.
2025-11-30T08:30:19.207791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-30T08:30:19.207807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/8] d0000000000_00000000000000000016_00000_0000000001_00002?
2025-11-30T08:30:19.207816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:19.207820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000016_00000_0000000001_00002?, parameters.CurOffset=16
2025-11-30T08:30:19.207994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 16.0/3
2025-11-30T08:30:19.208016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 0
2025-11-30T08:30:19.208024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 16:0 appended
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 23
    Offset: 23
    PartNo: 0
    WriteTimestampMS: 1764491419195
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 11
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
2025-11-30T08:30:19.208183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 16.1/3
2025-11-30T08:30:19.208197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 1
2025-11-30T08:30:19.208199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 16:1 appended
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:19.208217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 16.2/3
2025-11-30T08:30:19.208227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 2
2025-11-30T08:30:19.208841Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.208462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 16 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 1048790 count 1 nextOffset 17 batches 3
2025-11-30T08:30:19.208480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 16:2 appended
2025-11-30T08:30:19.208485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/8] d0000000000_00000000000000000017_00000_0000000001_00002?
2025-11-30T08:30:19.208488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:19.208491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000017_00000_0000000001_00002?, parameters.CurOffset=17
2025-11-30T08:30:19.208649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 17.0/3
2025-11-30T08:30:19.208655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 0
2025-11-30T08:30:19.208658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 17:0 appended
2025-11-30T08:30:19.208837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 17.1/3
2025-11-30T08:30:19.208842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 1
2025-11-30T08:30:19.208847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 17:1 appended
2025-11-30T08:30:19.208863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 17.2/3
2025-11-30T08:30:19.208866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 2
2025-11-30T08:30:19.208873Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:19.208881Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:19.208886Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:19.208893Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:19.208897Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:19.208900Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:19.208904Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:19.208908Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:19.209044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 17 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 2097545 count 2 nextOffset 18 batches 5
2025-11-30T08:30:19.209048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 17:2 appended
2025-11-30T08:30:19.209052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/8] d0000000000_00000000000000000018_00000_0000000001_00002?
2025-11-30T08:30:19.209055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:19.209058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000018_00000_0000000001_00002?, parameters.CurOffset=18
2025-11-30T08:30:19.209192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 18.0/3
2025-11-30T08:30:19.209196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 0
2025-11-30T08:30:19.209198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 18:0 appended
2025-11-30T08:30:19.209335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 18.1/3
2025-11-30T08:30:19.209339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 1
2025-11-30T08:30:19.209341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 18:1 appended
2025-11-30T08:30:19.209356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 18.2/3
2025-11-30T08:30:19.209359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 2
2025-11-30T08:30:19.209531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 18 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 3146300 count 3 nextOffset 19 batches 7
2025-11-30T08:30:19.209536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 18:2 appended
2025-11-30T08:30:19.209540Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/8] d0000000000_00000000000000000019_00000_0000000001_00002?
2025-11-30T08:30:19.209542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:19.209545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000019_00000_0000000001_00002?, parameters.CurOffset=19
2025-11-30T08:30:19.209685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 19.0/3
2025-11-30T08:30:19.209690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 0
2025-11-30T08:30:19.209692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 19:0 appended
2025-11-30T08:30:19.209845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 19.1/3
2025-11-30T08:30:19.209850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 1
2025-11-30T08:30:19.209852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 19:1 appended
2025-11-30T08:30:19.209879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 19.2/3
2025-11-30T08:30:19.209882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 2
2025-11-30T08:30:19.210061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 19 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 4195055 count 4 nextOffset 20 batches 9
2025-11-30T08:30:19.210065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 19:2 appended
2025-11-30T08:30:19.210070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/8] d0000000000_00000000000000000020_00000_0000000001_00002?
2025-11-30T08:30:19.210072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:19.210075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000020_00000_0000000001_00002?, parameters.CurOffset=20
2025-11-30T08:30:19.210216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 20.0/3
2025-11-30T08:30:19.210219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 0
2025-11-30T08:30:19.210221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 20:0 appended
2025-11-30T08:30:19.210363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 20.1/3
2025-11-30T08:30:19.210366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 1
2025-11-30T08:30:19.210368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 20:1 appended
2025-11-30T08:30:19.210377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 20.2/3
2025-11-30T08:30:19.210382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 2
2025-11-30T08:30:19.210569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 20 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 5243810 count 5 nextOffset 21 batches 11
2025-11-30T08:30:19.210573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 20:2 appended
2025-11-30T08:30:19.210577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/8] d0000000000_00000000000000000021_00000_0000000001_00002?
2025-11-30T08:30:19.210579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:19.210583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000021_00000_0000000001_00002?, parameters.CurOffset=21
2025-11-30T08:30:19.210729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 21.0/3
2025-11-30T08:30:19.210733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 0
2025-11-30T08:30:19.210735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 21:0 appended
2025-11-30T08:30:19.210883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 21.1/3
2025-11-30T08:30:19.210887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 1
2025-11-30T08:30:19.210889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 21:1 appended
2025-11-30T08:30:19.210900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 21.2/3
2025-11-30T08:30:19.210903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 2
2025-11-30T08:30:19.211280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 21 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 6292565 count 6 nextOffset 22 batches 13
2025-11-30T08:30:19.211285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 21:2 appended
2025-11-30T08:30:19.211289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/8] d0000000000_00000000000000000022_00000_0000000001_00002?
2025-11-30T08:30:19.211291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:19.211293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000022_00000_0000000001_00002?, parameters.CurOffset=22
2025-11-30T08:30:19.212139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 22.0/3
2025-11-30T08:30:19.212152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 0
2025-11-30T08:30:19.212156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 22:0 appended
2025-11-30T08:30:19.212294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 22.1/3
2025-11-30T08:30:19.212297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 1
2025-11-30T08:30:19.212299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 22:1 appended
2025-11-30T08:30:19.212308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 22.2/3
2025-11-30T08:30:19.212311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 2
2025-11-30T08:30:19.212492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 22 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 7341320 count 7 nextOffset 23 batches 15
2025-11-30T08:30:19.212496Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 22:2 appended
2025-11-30T08:30:19.212500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/8] d0000000000_00000000000000000023_00000_0000000001_00002?
2025-11-30T08:30:19.212503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:19.212506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000023_00000_0000000001_00002?, parameters.CurOffset=23
2025-11-30T08:30:19.212650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 23.0/3
2025-11-30T08:30:19.212654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 0
2025-11-30T08:30:19.219221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 23 partNo 0 result is x0000000000_00000000000000000015_00001_0000000008_00015 size 7878063
2025-11-30T08:30:19.219245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 23:0 appended
2025-11-30T08:30:19.219582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 23.1/3
2025-11-30T08:30:19.219600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 1
2025-11-30T08:30:19.219604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 23:1 appended
2025-11-30T08:30:19.219622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 23.2/3
2025-11-30T08:30:19.219625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 2
2025-11-30T08:30:19.219640Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1106: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000015_00001_0000000008_00015 new key d0000000000_00000000000000000015_00001_0000000008_00015 size 7878063 WTime 1764491419219
2025-11-30T08:30:19.219886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 23 partNo 2 FormedBlobsCount 1 NewHead: Offset 23 PartNo 0 PackedSize 1048790 count 1 nextOffset 24 batches 3
2025-11-30T08:30:19.219908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 23:2 appended
2025-11-30T08:30:19.220587Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:19.219972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:636: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 23,1 HeadOffset 15 endOffset 16 curOffset 24 d0000000000_00000000000000000023_00000_0000000001_00002| size 1048780 WTime 1764491419219
2025-11-30T08:30:19.220161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723642:3482] destroyed
2025-11-30T08:30:19.220171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723647:3485], now have 1 active actors on pipe
2025-11-30T08:30:19.220200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.220207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.220225Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:19.220238Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 15 partNo 1 count 8 size 7878063
2025-11-30T08:30:19.220246Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 23 partNo 0 count 1 size 1048780
2025-11-30T08:30:19.220287Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ef4cd215-9fe6def9-edd65da6-431e9bd1_24 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:19.220298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.220303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.220310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:19.220312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.220315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.220328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:19.220332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:19.220335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.220357Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:19.220413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.221078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723647:3485] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ef4cd215-9fe6def9-edd65da6-431e9bd1_24"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ef4cd215-9fe6def9-edd65da6-431e9bd1_24"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 24
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:19.228560Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.229163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723651:3487], now have 1 active actors on pipe
2025-11-30T08:30:19.233985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.234011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.234107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 24 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:19.234164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 24 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:19.234176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 24 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:19.234181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 24 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:19.234242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.234247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.234250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.234254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.234258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.234275Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:19.234291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.234295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.234306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.234311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.234314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:19.234316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.234320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.234360Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 0
2025-11-30T08:30:19.234376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.234379Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 1
2025-11-30T08:30:19.234383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.234386Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 2
2025-11-30T08:30:19.234631Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 24 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 1048790 count 1 nextOffset 25 batches 3
2025-11-30T08:30:19.234642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:19.234646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:19.234649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.234729Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 24,1 HeadOffset 24 endOffset 24 curOffset 25 d0000000000_00000000000000000024_00000_0000000001_00002? size 1048780 WTime 1764491419234
2025-11-30T08:30:19.235193Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:19.235210Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 24 partNo 0 count 1 size 1048780
2025-11-30T08:30:19.240989Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 15 count 8 size 7878063 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.241010Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 23 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.241084Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 15 partno 1 count 8 parts 15 suffix '0' size 7878063
2025-11-30T08:30:19.241096Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 7 parts 16 suffix '0' size 8365339
2025-11-30T08:30:19.241099Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 2 suffix '124' size 1048780
2025-11-30T08:30:19.241225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:19.241451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2279: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-30T08:30:19.241454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:19.241468Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 0 size 8365339 cause it's been evicted from L2. Actual L1 size: 12
2025-11-30T08:30:19.241475Z node 2 :PERSQUEUE NOTICE: read.h:372: [72075186224037892][PQCacheProxy]Have to remove new data from cache. Topic rt3.dc1--topic1, tablet id72075186224037892, cookie 0
2025-11-30T08:30:19.245380Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:19.244925Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 24 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.244956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:19.244987Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:19.244997Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.245011Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 24, partNo: 0, Offset: 24 is stored on disk
2025-11-30T08:30:19.245018Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.245024Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 24, partNo: 1, Offset: 24 is stored on disk
2025-11-30T08:30:19.245031Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.245036Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 24, partNo: 2, Offset: 24 is stored on disk
2025-11-30T08:30:19.245083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.245087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.245089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.245093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.245095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.245105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000015_00001_0000000001_00001|
2025-11-30T08:30:19.245111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000016_00000_0000000001_00002?
2025-11-30T08:30:19.245114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000017_00000_0000000001_00002?
2025-11-30T08:30:19.245118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000018_00000_0000000001_00002?
2025-11-30T08:30:19.245128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000019_00000_0000000001_00002?
2025-11-30T08:30:19.245132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000020_00000_0000000001_00002?
2025-11-30T08:30:19.245136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000021_00000_0000000001_00002?
2025-11-30T08:30:19.245140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000022_00000_0000000001_00002?
2025-11-30T08:30:19.245145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000023_00000_0000000001_00002?
2025-11-30T08:30:19.245171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:19.245193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.245214Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:19.245224Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000015_00001_0000000001_00001|(+) to d0000000000_00000000000000000015_00001_0000000001_00001|(+)
2025-11-30T08:30:19.245226Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000016_00000_0000000001_00002?(+) to d0000000000_00000000000000000016_00000_0000000001_00002?(+)
2025-11-30T08:30:19.245232Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000017_00000_0000000001_00002?(+) to d0000000000_00000000000000000017_00000_0000000001_00002?(+)
2025-11-30T08:30:19.245235Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000018_00000_0000000001_00002?(+) to d0000000000_00000000000000000018_00000_0000000001_00002?(+)
2025-11-30T08:30:19.245253Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000019_00000_0000000001_00002?(+) to d0000000000_00000000000000000019_00000_0000000001_00002?(+)
2025-11-30T08:30:19.245256Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000020_00000_0000000001_00002?(+) to d0000000000_00000000000000000020_00000_0000000001_00002?(+)
2025-11-30T08:30:19.245258Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000021_00000_0000000001_00002?(+) to d0000000000_00000000000000000021_00000_0000000001_00002?(+)
2025-11-30T08:30:19.245261Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000022_00000_0000000001_00002?(+) to d0000000000_00000000000000000022_00000_0000000001_00002?(+)
2025-11-30T08:30:19.245264Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000023_00000_0000000001_00002?(+) to d0000000000_00000000000000000023_00000_0000000001_00002?(+)
2025-11-30T08:30:19.245264Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.245835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723651:3487] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 24
    Offset: 24
    PartNo: 0
    WriteTimestampMS: 1764491419234
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 10
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:19.246785Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.249760Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|e5fff40f-9bc1293b-8b2a4c3f-dfeeed7b_25"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:12447
2025-11-30T08:30:19.249175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723659:3493], now have 1 active actors on pipe
2025-11-30T08:30:19.249316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 15 count 1 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.249322Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 16 count 1 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.249330Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 17 count 1 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.249334Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 18 count 1 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.249342Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 19 count 1 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.249347Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 20 count 1 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.249353Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 21 count 1 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.249358Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 22 count 1 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.249364Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 23 count 1 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.249404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.249408Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 15 partno 1 count 1 parts 1 suffix '124' size 536753
2025-11-30T08:30:19.249413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.249429Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.249432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:19.249443Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.249452Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:19.249459Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.249464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.249465Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.249467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.249470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.249471Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.249473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.249476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.249478Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.249486Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.249489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:19.249492Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.249515Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|e5fff40f-9bc1293b-8b2a4c3f-dfeeed7b_25 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:19.249523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.249528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.249533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:19.249535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.249538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.249550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:19.249553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:19.249556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.249568Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:19.249582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.250534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723659:3493] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|e5fff40f-9bc1293b-8b2a4c3f-dfeeed7b_25"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 25
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:19.257057Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.257688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723663:3495], now have 1 active actors on pipe
2025-11-30T08:30:19.259016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.259030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.259124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 25 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:19.259182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 25 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:19.259194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 25 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:19.259199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 25 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:19.259225Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:19.259250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.259277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.259290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.259294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.259299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:19.259302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.259307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.259348Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 0
2025-11-30T08:30:19.259363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.259367Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 1
2025-11-30T08:30:19.259372Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.259375Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 2
2025-11-30T08:30:19.259630Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 25 partNo 2 FormedBlobsCount 0 NewHead: Offset 25 PartNo 0 PackedSize 1048790 count 1 nextOffset 26 batches 3
2025-11-30T08:30:19.259642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:19.259646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:19.259649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.259747Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 25,1 HeadOffset 25 endOffset 25 curOffset 26 d0000000000_00000000000000000025_00000_0000000001_00002? size 1048780 WTime 1764491419259
2025-11-30T08:30:19.259940Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:19.259953Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 25 partNo 0 count 1 size 1048780
2025-11-30T08:30:19.263109Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 25 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.263144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:19.263172Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:19.263170Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.263181Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.263195Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 25, partNo: 0, Offset: 25 is stored on disk
2025-11-30T08:30:19.263200Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.263205Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 25, partNo: 1, Offset: 25 is stored on disk
2025-11-30T08:30:19.263211Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.263215Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 25, partNo: 2, Offset: 25 is stored on disk
2025-11-30T08:30:19.263255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.263258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.263261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.263264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.263266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.263276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:19.263413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.263516Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:19.263911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723663:3495] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 25
    Offset: 25
    PartNo: 0
    WriteTimestampMS: 1764491419259
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:19.264845Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.265328Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723668:3498], now have 1 active actors on pipe
2025-11-30T08:30:19.265432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.265453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.265492Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|af664ae6-18ac834f-e75ed97-8b477b09_26 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:19.265511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.265514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.265519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:19.265521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.265524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.265533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:19.265535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:19.265538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.265555Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:19.265570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.265656Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|af664ae6-18ac834f-e75ed97-8b477b09_26"
    Status: Active
    SeqNo: 0
  }
}
2025-11-30T08:30:19.265925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723668:3498] destroyed
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|af664ae6-18ac834f-e75ed97-8b477b09_26"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 26
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:19.272465Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.273031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723672:3500], now have 1 active actors on pipe
2025-11-30T08:30:19.274413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.274443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.274528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 26 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:19.274648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 26 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:19.274667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 26 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:19.274672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 26 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:19.274701Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:19.274733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.274737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.274749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.274752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.274757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:19.274760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.274766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.274805Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 0
2025-11-30T08:30:19.274820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.274824Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 1
2025-11-30T08:30:19.274828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.274831Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 2
2025-11-30T08:30:19.275089Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 26 partNo 2 FormedBlobsCount 0 NewHead: Offset 26 PartNo 0 PackedSize 1048790 count 1 nextOffset 27 batches 3
2025-11-30T08:30:19.275108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:19.275112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:19.275116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.275198Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 26,1 HeadOffset 26 endOffset 26 curOffset 27 d0000000000_00000000000000000026_00000_0000000001_00002? size 1048780 WTime 1764491419273
2025-11-30T08:30:19.275399Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:19.275423Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 26 partNo 0 count 1 size 1048780
2025-11-30T08:30:19.278394Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 26 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.278416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:19.278446Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:19.278454Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.278465Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 26, partNo: 0, Offset: 26 is stored on disk
2025-11-30T08:30:19.278469Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.278477Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 26, partNo: 1, Offset: 26 is stored on disk
2025-11-30T08:30:19.278484Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.278489Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 26, partNo: 2, Offset: 26 is stored on disk
2025-11-30T08:30:19.278529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.278532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.278535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.278537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.278540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.278550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:19.278570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.278623Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.278725Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:19.279224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723672:3500] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 26
    Offset: 26
    PartNo: 0
    WriteTimestampMS: 1764491419273
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:19.281218Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.281810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723677:3503], now have 1 active actors on pipe
2025-11-30T08:30:19.282036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.282047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.282084Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|bb76eeea-56fa1211-a669df10-d15cbf06_27 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:19.282095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.282100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.282108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:19.282110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.282115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.282126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:19.282129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:19.282133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.282151Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:19.282169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.282333Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|bb76eeea-56fa1211-a669df10-d15cbf06_27"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|bb76eeea-56fa1211-a669df10-d15cbf06_27"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 27
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:19.292935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723677:3503] destroyed
2025-11-30T08:30:19.295223Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.297236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723681:3505], now have 1 active actors on pipe
2025-11-30T08:30:19.297682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.297702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.297779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 27 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:19.297840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 27 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:19.297857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 27 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:19.297863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 27 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:19.297896Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:19.297922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.297926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.297956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.297963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.297969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:19.297971Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.297976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.298022Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 0
2025-11-30T08:30:19.298037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.298041Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 1
2025-11-30T08:30:19.298045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.298048Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 2
2025-11-30T08:30:19.298322Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 27 partNo 2 FormedBlobsCount 0 NewHead: Offset 27 PartNo 0 PackedSize 1048790 count 1 nextOffset 28 batches 3
2025-11-30T08:30:19.298334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:19.298338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:19.298342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.298420Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 27,1 HeadOffset 27 endOffset 27 curOffset 28 d0000000000_00000000000000000027_00000_0000000001_00002? size 1048780 WTime 1764491419295
2025-11-30T08:30:19.298634Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:19.298654Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 27 partNo 0 count 1 size 1048780
2025-11-30T08:30:19.303372Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 27 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.303398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:19.303424Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:19.303432Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.303443Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 27, partNo: 0, Offset: 27 is stored on disk
2025-11-30T08:30:19.303448Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.303453Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 27, partNo: 1, Offset: 27 is stored on disk
2025-11-30T08:30:19.303460Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.303464Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 27, partNo: 2, Offset: 27 is stored on disk
2025-11-30T08:30:19.303506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.303512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.303515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.303518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.303521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.303534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:19.303554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.303610Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.303835Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:19.304410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723681:3505] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 27
    Offset: 27
    PartNo: 0
    WriteTimestampMS: 1764491419295
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:19.305522Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.306120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723687:3508], now have 1 active actors on pipe
2025-11-30T08:30:19.306304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.306318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.306353Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|d1798178-ff5f6dac-1fc146d6-6f9d78d5_28 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:19.306364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.306369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.306375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:19.306378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.306381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.306393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:19.306396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:19.306399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.306419Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:19.306436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.306603Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:19.307120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723687:3508] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|d1798178-ff5f6dac-1fc146d6-6f9d78d5_28"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|d1798178-ff5f6dac-1fc146d6-6f9d78d5_28"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 28
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:19.314148Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.315010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723690:3509], now have 1 active actors on pipe
2025-11-30T08:30:19.316435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.316458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.316547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 28 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:19.316636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 28 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:19.316649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 28 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:19.316659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 28 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:19.316684Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:19.316708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.316713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.316726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.316730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.316735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:19.316737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.316743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.316784Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 0
2025-11-30T08:30:19.316823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.316827Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 1
2025-11-30T08:30:19.316831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.316836Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 2
2025-11-30T08:30:19.317068Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 28 partNo 2 FormedBlobsCount 0 NewHead: Offset 28 PartNo 0 PackedSize 1048790 count 1 nextOffset 29 batches 3
2025-11-30T08:30:19.317082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:19.317085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:19.317089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.317173Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 28,1 HeadOffset 28 endOffset 28 curOffset 29 d0000000000_00000000000000000028_00000_0000000001_00002? size 1048780 WTime 1764491419316
2025-11-30T08:30:19.317531Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:19.317557Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 28 partNo 0 count 1 size 1048780
2025-11-30T08:30:19.321441Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 28 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.321467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:19.321496Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:19.321504Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.321516Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 28, partNo: 0, Offset: 28 is stored on disk
2025-11-30T08:30:19.321521Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.321526Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 28, partNo: 1, Offset: 28 is stored on disk
2025-11-30T08:30:19.321533Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.321538Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 28, partNo: 2, Offset: 28 is stored on disk
2025-11-30T08:30:19.321581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.321585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.321587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.321590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.321593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.321608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:19.321628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.321774Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.321883Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 28
    Offset: 28
    PartNo: 0
    WriteTimestampMS: 1764491419313
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 8
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:19.323314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723690:3509] destroyed
2025-11-30T08:30:19.323419Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.324023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723695:3512], now have 1 active actors on pipe
2025-11-30T08:30:19.324120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.324146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.324201Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|cbb7da3f-1db7c071-da83d462-fcb70807_29 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:19.324217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.324221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.324228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:19.324230Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.324234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.324246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:19.324249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:19.324253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.324272Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:19.324295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.324364Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|cbb7da3f-1db7c071-da83d462-fcb70807_29"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:12447
2025-11-30T08:30:19.325941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723695:3512] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|cbb7da3f-1db7c071-da83d462-fcb70807_29"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 29
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:19.332379Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.333184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723699:3514], now have 1 active actors on pipe
2025-11-30T08:30:19.334491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.334516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.334648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 29 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:19.334705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 29 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:19.334718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 29 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:19.334723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 29 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:19.334747Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:19.334771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.334776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.334788Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.334792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.334797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:19.334800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.334806Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.334847Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 0
2025-11-30T08:30:19.334864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.334869Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 1
2025-11-30T08:30:19.334873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.334876Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 2
2025-11-30T08:30:19.335509Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 29 partNo 2 FormedBlobsCount 0 NewHead: Offset 29 PartNo 0 PackedSize 1048790 count 1 nextOffset 30 batches 3
2025-11-30T08:30:19.335523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:19.335526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:19.335531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.335613Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 29,1 HeadOffset 29 endOffset 29 curOffset 30 d0000000000_00000000000000000029_00000_0000000001_00002? size 1048780 WTime 1764491419335
2025-11-30T08:30:19.335835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2279: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-30T08:30:19.335844Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:19.335857Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 29 partNo 0 count 1 size 1048780
2025-11-30T08:30:19.341300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 29 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.341329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:19.341359Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:19.341368Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.341383Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 29, partNo: 0, Offset: 29 is stored on disk
2025-11-30T08:30:19.341388Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.341394Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 29, partNo: 1, Offset: 29 is stored on disk
2025-11-30T08:30:19.341402Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.341408Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 29, partNo: 2, Offset: 29 is stored on disk
2025-11-30T08:30:19.341458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.341462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.341464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.341468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.341470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.341484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:19.341506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.341687Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.341777Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 29
    Offset: 29
    PartNo: 0
    WriteTimestampMS: 1764491419333
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 7
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:19.343218Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.343236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723699:3514] destroyed
2025-11-30T08:30:19.343874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723704:3517], now have 1 active actors on pipe
2025-11-30T08:30:19.343892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.343900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.343936Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|14cf57f7-59d0d384-a7887f1e-44ef3f43_30 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:19.343952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.343957Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.343964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:19.343966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.343970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.343981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:19.343985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:19.343988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.344006Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:19.344022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.344196Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|14cf57f7-59d0d384-a7887f1e-44ef3f43_30"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|14cf57f7-59d0d384-a7887f1e-44ef3f43_30"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 30
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:19.351624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723704:3517] destroyed
2025-11-30T08:30:19.352446Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.352999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723707:3518], now have 1 active actors on pipe
2025-11-30T08:30:19.354281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.354302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.354391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 30 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:19.354523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 30 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:19.354536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 30 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:19.354541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 30 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:19.354563Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:19.354588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.354593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.354606Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.354609Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.354613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:19.354617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.354622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.354662Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 0
2025-11-30T08:30:19.354677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.354680Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 1
2025-11-30T08:30:19.354683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.354687Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 2
2025-11-30T08:30:19.355317Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 30 partNo 2 FormedBlobsCount 0 NewHead: Offset 30 PartNo 0 PackedSize 1048790 count 1 nextOffset 31 batches 3
2025-11-30T08:30:19.355332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:19.355338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:19.355345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.355428Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 30,1 HeadOffset 30 endOffset 30 curOffset 31 d0000000000_00000000000000000030_00000_0000000001_00002? size 1048780 WTime 1764491419352
2025-11-30T08:30:19.355638Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:19.355653Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 30 partNo 0 count 1 size 1048780
2025-11-30T08:30:19.359323Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 30 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.359351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:19.359363Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.359379Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:19.359387Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.359399Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 30, partNo: 0, Offset: 30 is stored on disk
2025-11-30T08:30:19.359409Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.359413Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 30, partNo: 1, Offset: 30 is stored on disk
2025-11-30T08:30:19.359421Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.359430Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 30, partNo: 2, Offset: 30 is stored on disk
2025-11-30T08:30:19.359483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.359492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.359495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.359498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.359502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.359514Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:19.359538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.359719Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:19.360129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723707:3518] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 30
    Offset: 30
    PartNo: 0
    WriteTimestampMS: 1764491419352
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:30:19.361485Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.362104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723712:3521], now have 1 active actors on pipe
2025-11-30T08:30:19.363055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.363070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.364103Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|66197008-e79a1d00-ddae80aa-358c22c6_31 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:30:19.364120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.364125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.364132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:30:19.364134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.364141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.364154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:30:19.364158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:30:19.364162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.364183Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:30:19.364206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.364703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723712:3521] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|66197008-e79a1d00-ddae80aa-358c22c6_31"
    Status: Active
    SeqNo: 0
  }
}
2025-11-30T08:30:19.364367Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|66197008-e79a1d00-ddae80aa-358c22c6_31"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 31
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:19.373789Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.374612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723715:3522], now have 1 active actors on pipe
2025-11-30T08:30:19.376336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.376359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.376439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 31 partNo : 0 messageNo: 0 size: 511959
2025-11-30T08:30:19.376518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 31 partNo : 1 messageNo: 0 size: 511959
2025-11-30T08:30:19.376530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 31 partNo : 2 messageNo: 0 size: 24658
2025-11-30T08:30:19.376535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 31 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-30T08:30:19.376560Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:30:19.376582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.376587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.376599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.376603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:30:19.376607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-30T08:30:19.376609Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.376614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.376652Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 0
2025-11-30T08:30:19.376667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.376671Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 1
2025-11-30T08:30:19.376674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:30:19.376679Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 2
2025-11-30T08:30:19.377068Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 31 partNo 2 FormedBlobsCount 0 NewHead: Offset 31 PartNo 0 PackedSize 1048790 count 1 nextOffset 32 batches 3
2025-11-30T08:30:19.377086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-30T08:30:19.377089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-30T08:30:19.377092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.377164Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 31,1 HeadOffset 31 endOffset 31 curOffset 32 d0000000000_00000000000000000031_00000_0000000001_00002? size 1048780 WTime 1764491419376
2025-11-30T08:30:19.377362Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:19.377376Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 31 partNo 0 count 1 size 1048780
2025-11-30T08:30:19.385051Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 31 count 1 size 1048780 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.385094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:19.385130Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:19.385141Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.385170Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 31, partNo: 0, Offset: 31 is stored on disk
2025-11-30T08:30:19.385175Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.385181Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 31, partNo: 1, Offset: 31 is stored on disk
2025-11-30T08:30:19.385188Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:30:19.385195Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 31, partNo: 2, Offset: 31 is stored on disk
2025-11-30T08:30:19.385253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.385257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.385260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.385266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.385269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.385285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000024_00000_0000000001_00002? (1048780)
2025-11-30T08:30:19.385289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000025_00000_0000000001_00002? (1048780)
2025-11-30T08:30:19.385293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000026_00000_0000000001_00002? (1048780)
2025-11-30T08:30:19.385297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000027_00000_0000000001_00002? (1048780)
2025-11-30T08:30:19.385301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000028_00000_0000000001_00002? (1048780)
2025-11-30T08:30:19.385304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000029_00000_0000000001_00002? (1048780)
2025-11-30T08:30:19.385306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000030_00000_0000000001_00002? (1048780)
2025-11-30T08:30:19.385309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000031_00000_0000000001_00002? (1048780)
2025-11-30T08:30:19.385312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 8 keys were taken away. Let's read 8390240 bytes
2025-11-30T08:30:19.385335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.385363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 8 blobs
2025-11-30T08:30:19.385366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000024_00000_0000000001_00002?
2025-11-30T08:30:19.385378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000025_00000_0000000001_00002?
2025-11-30T08:30:19.385391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000026_00000_0000000001_00002?
2025-11-30T08:30:19.385405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000027_00000_0000000001_00002?
2025-11-30T08:30:19.385413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000028_00000_0000000001_00002?
2025-11-30T08:30:19.385430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000029_00000_0000000001_00002?
2025-11-30T08:30:19.385438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000030_00000_0000000001_00002?
2025-11-30T08:30:19.385446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000031_00000_0000000001_00002?
2025-11-30T08:30:19.385457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 8 blobs for compaction
2025-11-30T08:30:19.385470Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 24 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:19.000000Z
2025-11-30T08:30:19.385474Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 25 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:19.000000Z
2025-11-30T08:30:19.385479Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 26 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:19.000000Z
2025-11-30T08:30:19.385483Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 27 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:19.000000Z
2025-11-30T08:30:19.385487Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 28 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:19.000000Z
2025-11-30T08:30:19.385491Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 29 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:19.000000Z
2025-11-30T08:30:19.385495Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 30 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:19.000000Z
2025-11-30T08:30:19.385499Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 31 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-30T08:30:19.000000Z
2025-11-30T08:30:19.385508Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 8 blobs are from cache.
2025-11-30T08:30:19.385540Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-30T08:30:19.385551Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/8] d0000000000_00000000000000000024_00000_0000000001_00002?
2025-11-30T08:30:19.385553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:19.385557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000024_00000_0000000001_00002?, parameters.CurOffset=24
2025-11-30T08:30:19.385714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 24.0/3
2025-11-30T08:30:19.385724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 0
2025-11-30T08:30:19.385731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 24:0 appended
2025-11-30T08:30:19.385896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 24.1/3
2025-11-30T08:30:19.385901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 1
2025-11-30T08:30:19.385904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 24:1 appended
2025-11-30T08:30:19.385925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 24.2/3
2025-11-30T08:30:19.385942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 2
2025-11-30T08:30:19.386190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 24 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 1048790 count 1 nextOffset 25 batches 3
2025-11-30T08:30:19.386196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 24:2 appended
2025-11-30T08:30:19.386204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/8] d0000000000_00000000000000000025_00000_0000000001_00002?
2025-11-30T08:30:19.386207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:19.386210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000025_00000_0000000001_00002?, parameters.CurOffset=25
2025-11-30T08:30:19.386363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 25.0/3
2025-11-30T08:30:19.386370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 0
2025-11-30T08:30:19.386374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 25:0 appended
2025-11-30T08:30:19.386962Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 25.1/3
2025-11-30T08:30:19.386968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 1
2025-11-30T08:30:19.386970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 25:1 appended
2025-11-30T08:30:19.386987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 25.2/3
2025-11-30T08:30:19.386990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 2
2025-11-30T08:30:19.387187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 25 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 2097545 count 2 nextOffset 26 batches 5
2025-11-30T08:30:19.387192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 25:2 appended
2025-11-30T08:30:19.387196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/8] d0000000000_00000000000000000026_00000_0000000001_00002?
2025-11-30T08:30:19.387198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:19.387201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000026_00000_0000000001_00002?, parameters.CurOffset=26
2025-11-30T08:30:19.387349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 26.0/3
2025-11-30T08:30:19.387354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 0
2025-11-30T08:30:19.387357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 26:0 appended
2025-11-30T08:30:19.387922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 26.1/3
2025-11-30T08:30:19.387927Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 1
2025-11-30T08:30:19.387930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 26:1 appended
2025-11-30T08:30:19.387945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 26.2/3
2025-11-30T08:30:19.387948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 2
2025-11-30T08:30:19.388127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 26 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 3146300 count 3 nextOffset 27 batches 7
2025-11-30T08:30:19.388136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 26:2 appended
2025-11-30T08:30:19.388141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/8] d0000000000_00000000000000000027_00000_0000000001_00002?
2025-11-30T08:30:19.388143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:19.388145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000027_00000_0000000001_00002?, parameters.CurOffset=27
2025-11-30T08:30:19.388292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 27.0/3
2025-11-30T08:30:19.388298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 0
2025-11-30T08:30:19.388300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 27:0 appended
2025-11-30T08:30:19.388461Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.388475Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:19.388479Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:19.388483Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:19.388487Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:19.388491Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:19.388498Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:19.388502Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:19.388505Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 2 suffix '63'
2025-11-30T08:30:19.388745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 27.1/3
2025-11-30T08:30:19.388750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 1
2025-11-30T08:30:19.388754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 27:1 appended
2025-11-30T08:30:19.388770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 27.2/3
2025-11-30T08:30:19.388772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 2
2025-11-30T08:30:19.388941Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:19.388974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 27 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 4195055 count 4 nextOffset 28 batches 9
2025-11-30T08:30:19.388980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 27:2 appended
2025-11-30T08:30:19.388984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/8] d0000000000_00000000000000000028_00000_0000000001_00002?
2025-11-30T08:30:19.388987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:19.388989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000028_00000_0000000001_00002?, parameters.CurOffset=28
2025-11-30T08:30:19.389135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 28.0/3
2025-11-30T08:30:19.389151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 0
2025-11-30T08:30:19.389154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 28:0 appended
2025-11-30T08:30:19.389319Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 28.1/3
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 31
    Offset: 31
    PartNo: 0
    WriteTimestampMS: 1764491419373
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 11
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-30T08:30:19.389329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 1
2025-11-30T08:30:19.389332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 28:1 appended
2025-11-30T08:30:19.389347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 28.2/3
2025-11-30T08:30:19.389349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 2
2025-11-30T08:30:19.389566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 28 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 5243810 count 5 nextOffset 29 batches 11
2025-11-30T08:30:19.389573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 28:2 appended
2025-11-30T08:30:19.389577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/8] d0000000000_00000000000000000029_00000_0000000001_00002?
2025-11-30T08:30:19.389579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:19.389581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000029_00000_0000000001_00002?, parameters.CurOffset=29
2025-11-30T08:30:19.389721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 29.0/3
2025-11-30T08:30:19.389726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 0
2025-11-30T08:30:19.389728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 29:0 appended
>>>>> 1
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdRead {
    ClientId: "user"
    Offset: 0
    Count: 16
    ReadTimestampMs: 0
  }
}
2025-11-30T08:30:19.389867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 29.1/3
2025-11-30T08:30:19.389879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 1
2025-11-30T08:30:19.389881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 29:1 appended
2025-11-30T08:30:19.389896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 29.2/3
2025-11-30T08:30:19.389898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 2
2025-11-30T08:30:19.390096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 29 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 6292565 count 6 nextOffset 30 batches 13
2025-11-30T08:30:19.390103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 29:2 appended
2025-11-30T08:30:19.390107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/8] d0000000000_00000000000000000030_00000_0000000001_00002?
2025-11-30T08:30:19.390109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:19.390113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000030_00000_0000000001_00002?, parameters.CurOffset=30
2025-11-30T08:30:19.390253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 30.0/3
2025-11-30T08:30:19.390259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 0
2025-11-30T08:30:19.390262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 30:0 appended
2025-11-30T08:30:19.390411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 30.1/3
2025-11-30T08:30:19.390415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 1
2025-11-30T08:30:19.390417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 30:1 appended
2025-11-30T08:30:19.390438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 30.2/3
2025-11-30T08:30:19.390441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 2
2025-11-30T08:30:19.390635Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.393138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 30 partNo 2 result is x0000000000_00000000000000000023_00000_0000000007_00016 size 8365364
2025-11-30T08:30:19.393158Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1106: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000023_00000_0000000007_00016 new key d0000000000_00000000000000000023_00000_0000000007_00016 size 8365364 WTime 1764491419392
2025-11-30T08:30:19.393188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 30 partNo 2 FormedBlobsCount 1 NewHead: Offset 30 PartNo 2 PackedSize 24736 count 1 nextOffset 31 batches 1
2025-11-30T08:30:19.393193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 30:2 appended
2025-11-30T08:30:19.393198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/8] d0000000000_00000000000000000031_00000_0000000001_00002?
2025-11-30T08:30:19.393200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:30:19.393206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000031_00000_0000000001_00002?, parameters.CurOffset=31
2025-11-30T08:30:19.393330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 31.0/3
2025-11-30T08:30:19.393345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 0
2025-11-30T08:30:19.393347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 31:0 appended
2025-11-30T08:30:19.393489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 31.1/3
2025-11-30T08:30:19.393505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 1
2025-11-30T08:30:19.393510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 31:1 appended
2025-11-30T08:30:19.393523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 31.2/3
2025-11-30T08:30:19.393525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 2
2025-11-30T08:30:19.393739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 31 partNo 2 FormedBlobsCount 0 NewHead: Offset 30 PartNo 2 PackedSize 1073491 count 2 nextOffset 32 batches 3
2025-11-30T08:30:19.393751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 31:2 appended
2025-11-30T08:30:19.393818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:636: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 30,2 HeadOffset 23 endOffset 24 curOffset 32 d0000000000_00000000000000000030_00002_0000000002_00002| size 1073481 WTime 1764491419392
2025-11-30T08:30:19.393972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723715:3522] destroyed
2025-11-30T08:30:19.393982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723720:3525], now have 1 active actors on pipe
2025-11-30T08:30:19.394014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.394024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.394043Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:19.394055Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 23 partNo 0 count 7 size 8365364
2025-11-30T08:30:19.394063Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 30 partNo 2 count 2 size 1073481
2025-11-30T08:30:19.394172Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 4 Topic 'rt3.dc1--topic1' partition 0 user user offset 0 partno 0 count 16 size 26214400 endOffset 32 max time lag 0ms effective offset 0
2025-11-30T08:30:19.394382Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 4 added 3 blobs, size 25145475 count 24 last offset 15, current partition end offset: 32
2025-11-30T08:30:19.394387Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 4. Send blob request.
2025-11-30T08:30:19.394401Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 7 parts_count 16 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.394407Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 7 partno 2 count 8 parts_count 15 source 1 size 7878063 accessed 0 times before, last time 2025-11-30T08:30:19.000000Z
2025-11-30T08:30:19.394411Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 15 partno 1 count 8 parts_count 15 source 1 size 7878063 accessed 0 times before, last time 2025-11-30T08:30:19.000000Z
2025-11-30T08:30:19.394450Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 4. Have to read 1 of 3 from KV
2025-11-30T08:30:19.394639Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 7 partno 2 count 8 parts 15 suffix '0'
2025-11-30T08:30:19.394643Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 15 partno 1 count 8 parts 15 suffix '0'
2025-11-30T08:30:19.394648Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 7 parts_count 16
2025-11-30T08:30:19.428038Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 3 from KV. Status 1
2025-11-30T08:30:19.428059Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-11-30T08:30:19.428071Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 7 size 8365339 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.428157Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 2 suffix '124' size 1048780
2025-11-30T08:30:19.428163Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 7 parts 16 suffix '0' size 8365339
2025-11-30T08:30:19.428311Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 3 blobs
2025-11-30T08:30:19.428528Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.429389Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.429633Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-30T08:30:19.429924Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 1 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.430059Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 1 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-30T08:30:19.430198Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 2 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.430358Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 2 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-30T08:30:19.430481Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 3 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.430608Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 3 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-30T08:30:19.430745Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 4 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.430878Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 4 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-30T08:30:19.430999Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 5 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.431130Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 5 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-30T08:30:19.431263Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 6 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.431281Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 6 totakecount 7 count 1 size 24704 from pos 0 cbcount 1
2025-11-30T08:30:19.431520Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 7 totakecount 7 count 0 size 1024007 from pos 0 cbcount 2
2025-11-30T08:30:19.431552Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 7 totakecount 8 count 1 size 24704 from pos 0 cbcount 1
2025-11-30T08:30:19.431690Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 8 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.432147Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 8 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.432413Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 8 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-30T08:30:19.432558Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 9 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.432709Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 9 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-30T08:30:19.432887Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 10 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.433078Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 10 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-30T08:30:19.433244Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 11 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.433476Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 11 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-30T08:30:19.433635Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 12 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.433815Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 12 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-30T08:30:19.433983Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 13 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.434148Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 13 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-30T08:30:19.434300Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 14 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.434317Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 14 totakecount 8 count 1 size 24704 from pos 0 cbcount 1
2025-11-30T08:30:19.434458Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 15 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.434637Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 15 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.434651Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 15 totakecount 8 count 1 size 24704 from pos 0 cbcount 1
2025-11-30T08:30:19.434740Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 23 size 1048780 cause it's been evicted from L2. Actual L1 size: 11
2025-11-30T08:30:19.434748Z node 2 :PERSQUEUE NOTICE: read.h:372: [72075186224037892][PQCacheProxy]Have to remove new data from cache. Topic rt3.dc1--topic1, tablet id72075186224037892, cookie 0
2025-11-30T08:30:19.434867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.434897Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 23 count 7 size 8365364 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.434903Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 30 count 2 size 1073481 actorID [2:7578432936017424573:2363]
2025-11-30T08:30:19.434969Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.434978Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.434984Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.434989Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.434994Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.434999Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.435002Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.435008Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 7 parts 16 suffix '0' size 8365364
2025-11-30T08:30:19.435015Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-30T08:30:19.435020Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 7 partno 2 count 8 parts 15 suffix '0' size 7878063
2025-11-30T08:30:19.435024Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 30 partno 2 count 2 parts 2 suffix '124' size 1073481
2025-11-30T08:30:19.449538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:19.449764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.449780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.449783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.449787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.449791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.449803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000023_00000_0000000001_00002|
2025-11-30T08:30:19.449809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000024_00000_0000000001_00002?
2025-11-30T08:30:19.449813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000025_00000_0000000001_00002?
2025-11-30T08:30:19.449816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000026_00000_0000000001_00002?
2025-11-30T08:30:19.449820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000027_00000_0000000001_00002?
2025-11-30T08:30:19.449822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000028_00000_0000000001_00002?
2025-11-30T08:30:19.449828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000029_00000_0000000001_00002?
2025-11-30T08:30:19.449831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000030_00000_0000000001_00002?
2025-11-30T08:30:19.449835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000031_00000_0000000001_00002?
2025-11-30T08:30:19.449867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:19.449887Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 29 size 1048780 cause it's been evicted from L2. Actual L1 size: 12
2025-11-30T08:30:19.449891Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 26 size 1048780 cause it's been evicted from L2. Actual L1 size: 11
2025-11-30T08:30:19.449895Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 30 size 1048780 cause it's been evicted from L2. Actual L1 size: 10
2025-11-30T08:30:19.449899Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 25 size 1048780 cause it's been evicted from L2. Actual L1 size: 9
2025-11-30T08:30:19.449903Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 27 size 1048780 cause it's been evicted from L2. Actual L1 size: 8
2025-11-30T08:30:19.449906Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 31 size 1048780 cause it's been evicted from L2. Actual L1 size: 7
2025-11-30T08:30:19.449909Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 7 size 7878063 cause it's been evicted from L2. Actual L1 size: 6
2025-11-30T08:30:19.449911Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 24 size 1048780 cause it's been evicted from L2. Actual L1 size: 5
2025-11-30T08:30:19.449913Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 28 size 1048780 cause it's been evicted from L2. Actual L1 size: 4
2025-11-30T08:30:19.449920Z node 2 :PERSQUEUE NOTICE: read.h:372: [72075186224037892][PQCacheProxy]Have to remove new data from cache. Topic rt3.dc1--topic1, tablet id72075186224037892, cookie 0
2025-11-30T08:30:19.449945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2279: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-30T08:30:19.449950Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:30:19.449959Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000023_00000_0000000001_00002|(+) to d0000000000_00000000000000000023_00000_0000000001_00002|(+)
2025-11-30T08:30:19.449962Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000024_00000_0000000001_00002?(+) to d0000000000_00000000000000000024_00000_0000000001_00002?(+)
2025-11-30T08:30:19.449963Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000025_00000_0000000001_00002?(+) to d0000000000_00000000000000000025_00000_0000000001_00002?(+)
2025-11-30T08:30:19.449965Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000026_00000_0000000001_00002?(+) to d0000000000_00000000000000000026_00000_0000000001_00002?(+)
2025-11-30T08:30:19.449967Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000027_00000_0000000001_00002?(+) to d0000000000_00000000000000000027_00000_0000000001_00002?(+)
2025-11-30T08:30:19.449968Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000028_00000_0000000001_00002?(+) to d0000000000_00000000000000000028_00000_0000000001_00002?(+)
2025-11-30T08:30:19.449971Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000029_00000_0000000001_00002?(+) to d0000000000_00000000000000000029_00000_0000000001_00002?(+)
2025-11-30T08:30:19.449973Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000030_00000_0000000001_00002?(+) to d0000000000_00000000000000000030_00000_0000000001_00002?(+)
2025-11-30T08:30:19.449975Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000031_00000_0000000001_00002?(+) to d0000000000_00000000000000000031_00000_0000000001_00002?(+)
2025-11-30T08:30:19.463230Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:30:19.463269Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:30:19.463283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.463287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.463289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.463292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.463295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.463306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:30:19.463607Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:19.472434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723720:3525] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdReadResult {
    MaxOffset: 32
    Result {
      Offset: 0
      Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
>>>>> 2
CallPersQueueGRPC request to localhost:12447
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdRead {
    ClientId: "user"
    Offset: 16
    Count: 16
    ReadTimestampMs: 0
  }
}
2025-11-30T08:30:19.526763Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:30:19.527268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578432941979723734:3536], now have 1 active actors on pipe
2025-11-30T08:30:19.527374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:30:19.527387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:30:19.527435Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 5 Topic 'rt3.dc1--topic1' partition 0 user user offset 16 partno 0 count 16 size 26214400 endOffset 32 max time lag 0ms effective offset 16
2025-11-30T08:30:19.527740Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 5 added 2 blobs, size 9438782 count 16 last offset 23, current partition end offset: 32
2025-11-30T08:30:19.527750Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 5. Send blob request.
2025-11-30T08:30:19.527780Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 15 partno 1 count 8 parts_count 15 source 1 size 7878063 accessed 1 times before, last time 2025-11-30T08:30:19.000000Z
2025-11-30T08:30:19.527788Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 23 partno 0 count 7 parts_count 16 source 1 size 8365364 accessed 0 times before, last time 2025-11-30T08:30:19.000000Z
2025-11-30T08:30:19.527802Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 5. All 2 blobs are from cache.
2025-11-30T08:30:19.527838Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 2 blobs
2025-11-30T08:30:19.527845Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 15 partno 1 count 8 parts 15 suffix '0'
2025-11-30T08:30:19.527851Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 7 parts 16 suffix '0'
2025-11-30T08:30:19.528148Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 16 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.528390Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 16 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.528912Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 16 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-30T08:30:19.529120Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 17 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.529296Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 17 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-30T08:30:19.529575Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 18 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.529749Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 18 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-30T08:30:19.530056Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 19 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.530210Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 19 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-30T08:30:19.530472Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 20 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.530643Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 20 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-30T08:30:19.530793Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 21 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.530950Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 21 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-30T08:30:19.531116Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 22 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.531142Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 22 totakecount 8 count 1 size 24704 from pos 0 cbcount 1
2025-11-30T08:30:19.531348Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 23 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.531501Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 23 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.531518Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 23 totakecount 7 count 1 size 24704 from pos 0 cbcount 1
2025-11-30T08:30:19.532025Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 24 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.532201Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 24 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.532376Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 24 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-30T08:30:19.532530Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 25 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.532691Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 25 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-30T08:30:19.532968Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 26 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.533165Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 26 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-30T08:30:19.533322Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 27 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.533500Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 27 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-30T08:30:19.533665Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 28 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.533857Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 28 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-30T08:30:19.534213Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 29 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-30T08:30:19.534234Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 29 totakecount 7 count 1 size 24704 from pos 0 cbcount 1
2025-11-30T08:30:19.534759Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 30 totakecount 7 count 0 size 1024007 from pos 0 cbcount 2
2025-11-30T08:30:19.534883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:30:19.550033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.550058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.550061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.550065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.550068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.558100Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:30:19.569289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578432941979723734:3536] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdReadResult {
    MaxOffset: 32
    Result {
      Offset: 16
      Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-30T08:30:19.650424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.650442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.650444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.650448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.650452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.750845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.750868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.750873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.750877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.750879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.853123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.853144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.853147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.853152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.853156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:19.957100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:19.957117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.957120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:19.957123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:19.957125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:20.059444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:20.059466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:20.059468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:20.059472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:20.059475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:30:20.160057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:30:20.160075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:20.160078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:30:20.160082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:30:20.160084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
[[bad]]assertion failed at ydb/services/persqueue_v1/persqueue_ut.cpp:3269, virtual void NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TTestCaseCache::Execute_(NUnitTest::TTestContext &): (info0.BlobsFromCache == 1) failed: (2 != 1) [[rst]]
[[alt1]]TBackTrace::Capture()+28 (0x1514373C)
NUnitTest::NPrivate::RaiseError(char const*, TBasicString<char, std::__y1::char_traits<char>> const&, bool)+128 (0x152FF740)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TTestCaseCache::Execute_(NUnitTest::TTestContext&)+4375 (0x14E6DB67)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TCurrentTest::Execute()::'lambda'()::operator()() const+71 (0x14F5A6B7)
NUnitTest::TTestBase::Run(std::__y1::function<void ()>, TBasicString<char, std::__y1::char_traits<char>> const&, char const*, bool)+120 (0x15301918)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TCurrentTest::Execute()+436 (0x14F59FB4)
NUnitTest::TTestFactory::Execute()+817 (0x15302061)
NUnitTest::RunMain(int, char**)+3229 (0x153144CD)
??+0 (0x7F73AF98FD90)
__libc_start_main+128 (0x7F73AF98FE40)
_start+41 (0x13F57029)
[[rst]]
2025-11-30T08:31:06.738662Z node 1 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[1:7578433140633751352:2156];send_to=[0:7307199536658146131:7762515];
2025-11-30T08:31:06.738694Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
2025-11-30T08:31:06.743860Z node 2 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[2:7578433141311463082:2074];send_to=[0:7307199536658146131:7762515];
2025-11-30T08:31:06.743905Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
2025-11-30T08:31:06.743861Z node 1 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
test_client.cpp: SetPath # /home/runner/.ya/build/build_root/0ao2/000a1e/r3tmp/tmpmKFMTf/pdisk_1.dat
2025-11-30T08:31:06.747678Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-11-30T08:31:06.787458Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-11-30T08:31:06.789847Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-11-30T08:31:06.807853Z node 1 :IMPORT WARN: schemeshard_import.cpp:423: Table profiles were not loaded
TServer::EnableGrpc on GrpcPort 30859, node 1
2025-11-30T08:31:06.817259Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:190: distributable config is empty, broken or outdated, will use file: /home/runner/.ya/build/build_root/0ao2/000a1e/r3tmp/yandexyaQVE0.tmp
2025-11-30T08:31:06.817277Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:196: will try to initialize from file: /home/runner/.ya/build/build_root/0ao2/000a1e/r3tmp/yandexyaQVE0.tmp
2025-11-30T08:31:06.817354Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:202: successfully initialized from file: /home/runner/.ya/build/build_root/0ao2/000a1e/r3tmp/yandexyaQVE0.tmp
2025-11-30T08:31:06.817383Z node 1 :NET_CLASSIFIER ERROR: net_classifier.cpp:228: got bad distributable configuration
2025-11-30T08:31:06.821810Z INFO: TTestServer started on Port 1132 GrpcPort 30859
TClient is connected to server localhost:1132
PQClient connected to localhost:30859
WaitRootIsUp 'Root'...
TClient::Ls request: Root
TClient::Ls response: 2025-11-30T08:31:06.842232Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-11-30T08:31:06.842282Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Disconnected -> Connecting
Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "Root"
    PathId: 1
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 1
    CreateStep: 0
    ParentPathId: 1
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 1
      SubDomainVersion: 0
      SecurityStateVersion: 0
    }
    ChildrenExist: false
  }
  Children {
    Name: ".sys"
    PathId: 18446744073709551615
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 0
    CreateStep: 0
    ParentPathId: 18446744073709551615
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 0
      Pl...
(TRUNCATED)
WaitRootIsUp 'Root' success.
2025-11-30T08:31:06.845647Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-11-30T08:31:06.848315Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-11-30T08:31:06.848341Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-11-30T08:31:06.848868Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpAlterSubDomain, opId: 281474976720657:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_alter_subdomain.cpp:311)
waiting...
2025-11-30T08:31:06.851036Z node 1 :HIVE WARN: hive_impl.cpp:811: HIVE#72057594037968897 Handle TEvInterconnect::TEvNodeConnected, NodeId 2 Cookie 2
2025-11-30T08:31:06.851206Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Connecting -> Connected
waiting...
2025-11-30T08:31:06.889074Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720659:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
waiting...
waiting...
waiting...
2025-11-30T08:31:06.940585Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-11-30T08:31:06.975511Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-11-30T08:31:07.202520Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7578433144928719540:2326], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-30T08:31:07.202557Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7578433144928719566:2329], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-30T08:31:07.202568Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-30T08:31:07.202542Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7578433145606430732:2300], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-30T08:31:07.202574Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7578433145606430757:2303], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-30T08:31:07.202610Z node 2 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-30T08:31:07.202779Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7578433145606430762:2305], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-30T08:31:07.202807Z node 2 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-30T08:31:07.202984Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7578433144928719569:2331], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-30T08:31:07.203022Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-30T08:31:07.203511Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateResourcePool, opId: 281474976720662:3, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:179)
2025-11-30T08:31:07.204217Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7578433144928719605:2333], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-30T08:31:07.204235Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-30T08:31:07.204343Z node 1 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [1:7578433144928719608:2334], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-30T08:31:07.204354Z node 1 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-30T08:31:07.205304Z node 2 :TX_PROXY ERROR: schemereq.cpp:593: Actor# [2:7578433145606430763:2166] txid# 281474976715657, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exists but creating right now (id: [OwnerId: 72057594046644480, LocalPathId: 9], type: EPathTypeResourcePool, state: EPathStateCreate), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-11-30T08:31:07.210148Z node 1 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [1:7578433144928719568:2330], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976720662 completed, doublechecking }
2025-11-30T08:31:07.210197Z node 2 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [2:7578433145606430761:2304], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976720662 completed, doublechecking }
2025-11-30T08:31:07.259713Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720663:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
2025-11-30T08:31:07.278504Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720664:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
2025-11-30T08:31:07.278935Z node 2 :TX_PROXY ERROR: schemereq.cpp:593: Actor# [2:7578433145606430791:2174] txid# 281474976715658, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 9], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-11-30T08:31:07.294363Z node 1 :TX_PROXY ERROR: schemereq.cpp:593: Actor# [1:7578433144928719864:2894] txid# 281474976720665, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 9], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-11-30T08:31:07.300001Z node 2 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:625: Compilation failed, self: [2:7578433145606430805:2309], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:5:17: Error: At function: KiReadTable!
        <main>:5:17: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Versions]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-11-30T08:31:07.300930Z node 2 :KQP_SESSION WARN: kqp_session_actor.cpp:2542: SessionId: ydb://session/3?node_id=2&id=ZWM2NzkxYjktMTUzNTZkNDAtNTFjM2VkMTEtMjljZWEyMDE=, ActorId: [2:7578433145606430729:2298], ActorState: ExecuteState, TraceId: 01kb9y66dz2xaj0msevx84w3ee, ReplyQueryCompileError, status: SCHEME_ERROR, issues: { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 5 column: 17 } message: "At function: KiReadTable!" end_position { row: 5 column: 17 } severity: 1 issues { position { row: 5 column: 17 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Versions]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 5 column: 17 } issue_code: 2003 severity: 1 } } }, remove tx with tx_id: 
2025-11-30T08:31:07.301518Z node 2 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 5 column: 17 } message: "At function: KiReadTable!" end_position { row: 5 column: 17 } severity: 1 issues { position { row: 5 column: 17 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Versions]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 5 column: 17 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-11-30T08:31:07.300852Z node 1 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:625: Compilation failed, self: [1:7578433144928719899:2350], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:5:17: Error: At function: KiReadTable!
        <main>:5:17: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Versions]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-11-30T08:31:07.301032Z node 1 :KQP_SESSION WARN: kqp_session_actor.cpp:2542: SessionId: ydb://session/3?node_id=1&id=OWMxOGI4MTUtMzA3MGU5MWYtYjg4NmMwNzctYTc2OWJjYWM=, ActorId: [1:7578433144928719535:2323], ActorState: ExecuteState, TraceId: 01kb9y66dza0r1jf249qjzx7gy, ReplyQueryCompileError, status: SCHEME_ERROR, issues: { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 5 column: 17 } message: "At function: KiReadTable!" end_position { row: 5 column: 17 } severity: 1 issues { position { row: 5 column: 17 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Versions]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 5 column: 17 } issue_code: 2003 severity: 1 } } }, remove tx with tx_id: 
2025-11-30T08:31:07.301525Z node 1 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 5 column: 17 } message: "At function: KiReadTable!" end_position { row: 5 column: 17 } severity: 1 issues { position { row: 5 column: 17 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Versions]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 5 column: 17 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-11-30T08:31:07.307973Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720666:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
=== Init DC: UPSERT INTO `/Root/PQ/Config/V2/Cluster` (name, balancer, local, enabled, weight) VALUES ("dc1", "localhost", true, true, 1000), ("dc2", "dc2.logbroker.yandex.net", false, true, 1000);
=== CheckClustersList. Subcribe to ClusterTracker from [1:7578433144928720099:3054] 
2025-11-30T08:31:07.741479Z node 1 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-11-30T08:31:07.745567Z node 2 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-11-30T08:31:11.738776Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[1:7578433140633751352:2156];send_to=[0:7307199536658146131:7762515];
2025-11-30T08:31:11.738818Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
2025-11-30T08:31:11.744290Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[2:7578433141311463082:2074];send_to=[0:7307199536658146131:7762515];
2025-11-30T08:31:11.744336Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
=== CheckClustersList. Ok
PQ Client: create topic: rt3.dc1--topic1 with 1 partitions
CallPersQueueGRPC request to localhost:30859
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--topic1"
  }
}
2025-11-30T08:31:12.551966Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--topic1, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:30859
MetaRequest {
  CmdCreateTopic {
    Topic: "rt3.dc1--topic1"
    NumPartitions: 1
    Config {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 6291456
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
  }
}
CallPersQueueGRPC response:
Status: 129
ProxyErrorCode: 53
SchemeStatus: 1
FlatTxId {
  TxId: 281474976720673
  SchemeShardTabletId: 72057594046644480
  PathId: 13
}
ErrorCode: OK
AddTopic: rt3.dc1--topic1
===Run query:``DECLARE $version as Int64; DECLARE $path AS Utf8; DECLARE $cluster as Utf8; UPSERT INTO `/Root/PQ/Config/V2/Topics` (path, dc) VALUES ($path, $cluster); UPSERT INTO `/Root/PQ/Config/V2/Versions` (name, version) VALUES ("Topics", $version);`` with topic = topic1, dc = dc1
2025-11-30T08:31:12.568391Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1656: [72075186224037893][] pipe [1:7578433166403557014:3295] connected; active server actors: 1
2025-11-30T08:31:12.568604Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1520: [72075186224037893][rt3.dc1--topic1] updating configuration. Deleted partitions []. Added partitions [0]
2025-11-30T08:31:12.568760Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:942: [72075186224037893][rt3.dc1--topic1] Discovered subdomain [OwnerId: 72057594046644480, LocalPathId: 1] state, outOfSpace = 0 at RB 72075186224037893
2025-11-30T08:31:12.569257Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:140: [72075186224037893][rt3.dc1--topic1] BALANCER INIT DONE for rt3.dc1--topic1: (0, 72075186224037892) 
2025-11-30T08:31:12.580098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2959: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-11-30T08:31:12.581715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2991: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-11-30T08:31:12.581780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:787: [PQ: 72075186224037892] no config, start with empty partitions and default config
2025-11-30T08:31:12.581791Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:913: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-11-30T08:31:12.581801Z node 2 :PERSQUEUE INFO: pq_impl.cpp:612: [PQ: 72075186224037892] doesn't have tx writes info
2025-11-30T08:31:12.581926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433166403557034:2462], now have 1 active actors on pipe
2025-11-30T08:31:12.581934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433166403557013:3294], now have 1 active actors on pipe
2025-11-30T08:31:12.582272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3563: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-30T08:31:12.582473Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:404: [72075186224037893][rt3.dc1--topic1] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1
2025-11-30T08:31:12.583371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
===Query complete
TClient::Ls request: /Root/PQ/rt3.dc1--topic1
2025-11-30T08:31:12.590361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433166403557091:3348], now have 1 active actors on pipe
2025-11-30T08:31:12.590557Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:85: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitConfigStep
2025-11-30T08:31:12.590589Z node 2 :PERSQUEUE DEBUG: partition.cpp:1313: [72075186224037892][Partition][0][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-11-30T08:31:12.590682Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:85: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-11-30T08:31:12.590765Z node 2 :PERSQUEUE INFO: partition_init.cpp:1219: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7578433167081267559:2364]
2025-11-30T08:31:12.591438Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:85: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitFieldsStep
2025-11-30T08:31:12.591450Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:65: [rt3.dc1--topic1:0:Initializer] Initializing completed.
2025-11-30T08:31:12.591455Z node 2 :PERSQUEUE INFO: partition.cpp:705: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--topic1' partition 0 generation 1 [2:7578433167081267559:2364]
2025-11-30T08:31:12.591462Z node 2 :PERSQUEUE DEBUG: partition.cpp:725: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--topic1 partitition 0 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-11-30T08:31:12.591469Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:107: [72075186224037892][Partition][0][StateIdle] Initializing MLP Consumers: 0
2025-11-30T08:31:12.591745Z node 2 :PERSQUEUE DEBUG: partition.cpp:4399: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 1
2025-11-30T08:31:12.591758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:12.591773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2390: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TTransaction[ProposeConfig])
2025-11-30T08:31:12.591784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:12.591792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:12.591794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:12.591796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:12.591805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:12.591807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:12.591808Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:12.591810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:12.591812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:12.591821Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:930: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-11-30T08:31:12.591828Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:70: [72075186224037892][Partition][0][StateIdle] Process MLP pending events. Count 0
2025-11-30T08:31:12.591921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3563: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-30T08:31:12.591950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:12.592701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-30T08:31:12.592734Z node 2 :PERSQUEUE DEBUG: partition.cpp:1418: [72075186224037892][Partition][0][StateIdle] Handle TEvPQ::TEvTxCommit Step 1764491472639, TxId 281474976720673
2025-11-30T08:31:12.592741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:12.592743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:12.592745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:12.592750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2454: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TTransaction[ProposeConfig])
2025-11-30T08:31:12.592766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:12.592772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:12.592774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:12.592831Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:12.593436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:12.593474Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:107: [72075186224037892][Partition][0][StateIdle] Initializing MLP Consumers: 0
2025-11-30T08:31:12.593503Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:12.593513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:12.593516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:12.593517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:12.593520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:12.593521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:12.593526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:12.593623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:398: [PQ: 72075186224037892] Apply new config PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 6291456 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--topic1" Version: 0 LocalDC: true Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } TopicPath: "/Root/PQ/rt3.dc1--topic1" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Version: 0 Generation: 0 } MonitoringProjectId: ""
2025-11-30T08:31:12.593642Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:913: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-11-30T08:31:12.593678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3563: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-30T08:31:12.594164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-30T08:31:12.594189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3563: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-30T08:31:12.594811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
TClient::Ls response: Status: 128
StatusCode: PATH_NOT_EXIST
Issues {
  message: "Path not exist"
  issue_code: 200200
  severity: 1
}
SchemeStatus: 2
ErrorReason: "Path not found"
GetTopicVersionFromPath:  record Status: 128
StatusCode: PATH_NOT_EXIST
Issues {
  message: "Path not exist"
  issue_code: 200200
  severity: 1
}
SchemeStatus: 2
ErrorReason: "Path not found"
 name rt3.dc1--topic1 version0
2025-11-30T08:31:12.692011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:12.692029Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:12.692032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:12.692036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:12.692039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:12.796881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:12.796909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:12.796912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:12.796917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:12.796920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:12.900865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:12.900886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:12.900889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:12.900891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:12.900894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.000878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.000904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.000907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.000911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.000914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
TClient::Ls request: /Root/PQ/rt3.dc1--topic1
2025-11-30T08:31:13.101118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.101153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.101155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.101158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.101162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--topic1"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720673
    CreateStep: 1764491472639
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--topic1"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 6291456
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 600000...
(TRUNCATED)
GetTopicVersionFromPath:  record Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--topic1"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720673
    CreateStep: 1764491472639
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--topic1"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 6291456
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      YdbDatabasePath: "/Root"
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
    Partitions {
      PartitionId: 0
      TabletId: 72075186224037892
      Status: Active
    }
    AlterVersion: 1
    BalancerTabletID: 72075186224037893
    NextPartitionId: 1
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 1
      PlanResolution: 7
      Coordinators: 72057594046316545
      TimeCastBucketsPerMediator: 2
      Mediators: 72057594046382081
    }
    DomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    PathsInside: 12
    PathsLimit: 10000
    ShardsInside: 6
    ShardsLimit: 200000
    ResourcesDomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    DiskSpaceUsage {
      Tables {
        TotalSize: 0
        DataSize: 0
        IndexSize: 0
      }
      Topics {
        ReserveSize: 0
        AccountSize: 0
        DataSize: 0
        UsedReserveSize: 0
      }
    }
    PQPartitionsInside: 1
    PQPartitionsLimit: 1000000
    SchemeLimits {
      MaxDepth: 32
      MaxPaths: 10000
      MaxChildrenInDir: 100000
      MaxAclBytesSize: 10240
      MaxTableColumns: 200
      MaxTableColumnNameLength: 255
      MaxTableKeyColumns: 30
      MaxTableIndices: 20
      MaxShards: 200000
      MaxShardsInPath: 35000
      MaxConsistentCopyTargets: 10000
      MaxPathElementLength: 255
      ExtraPathSymbolsAllowed: "!\"#$%&\'()*+,-.:;<=>?@[\\]^_`{|}~"
      MaxPQPartitions: 1000000
      MaxTableCdcStreams: 5
      MaxExports: 10
      MaxImports: 10
      MaxColumnTableColumns: 10000
    }
  }
}
Path: "/Root/PQ/rt3.dc1--topic1"
 name rt3.dc1--topic1 version1
CallPersQueueGRPC request to localhost:30859
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--topic1"
  }
}
2025-11-30T08:31:13.104661Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
MetaResponse {
  CmdGetTopicMetadataResult {
    TopicInfo {
      Topic: "rt3.dc1--topic1"
      NumPartitions: 1
      Config {
        PartitionConfig {
          LifetimeSeconds: 86400
          LowWatermark: 6291456
          SourceIdLifetimeSeconds: 86400
          WriteSpeedInBytesPerSecond: 20000000
          BurstSize: 20000000
          SourceIdMaxCounts: 6000000
        }
        Version: 1
        LocalDC: true
        Codecs {
          Ids: 0
          Ids: 1
          Ids: 2
          Codecs: "raw"
          Codecs: "gzip"
          Codecs: "lzop"
        }
        TopicPath: "/Root/PQ/rt3.dc1--topic1"
        YdbDatabasePath: "/Root"
        Consumers {
          Name: "user"
          ReadFromTimestampsMs: 0
          FormatVersion: 0
          Version: 0
        }
      }
      ErrorCode: OK
    }
  }
}
=== Topic created, have version: 1
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.106853Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.107856Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|c3995397-dc7cffe7-2ff72821-8b891d2f_0"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.107376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524447:3391], now have 1 active actors on pipe
2025-11-30T08:31:13.107595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.107605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.107649Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|c3995397-dc7cffe7-2ff72821-8b891d2f_0 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.107664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.107668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.107675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.107677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.107683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.107695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.107698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.107701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.107721Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.107744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.108206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524447:3391] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|c3995397-dc7cffe7-2ff72821-8b891d2f_0"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 0
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.112304Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.112881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524450:3392], now have 1 active actors on pipe
2025-11-30T08:31:13.113294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.113304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.113401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 0 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.113415Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.113434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.113439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.113451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.113453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.113457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.115072Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 0
2025-11-30T08:31:13.115138Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 0 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 262214 count 1 nextOffset 1 batches 1
2025-11-30T08:31:13.115150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.115153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.115157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.115270Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 0,1 HeadOffset 0 endOffset 0 curOffset 1 d0000000000_00000000000000000000_00000_0000000001_00000? size 262204 WTime 1764491473112
2025-11-30T08:31:13.115328Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.115340Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 1 size 262204
2025-11-30T08:31:13.118565Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 0 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.118596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.118614Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.118626Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.118638Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.118651Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 0, partNo: 0, Offset: 0 is stored on disk
2025-11-30T08:31:13.118693Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:930: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-11-30T08:31:13.118699Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:972: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-11-30T08:31:13.118704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.118706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.118709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.118712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.118714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.118723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.118738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.118782Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--topic1' partition 0 user user offset 0 partno 0 count 1 size 1024000 endOffset 1 max time lag 0ms effective offset 0
2025-11-30T08:31:13.118798Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 262204 count 1 last offset 0, current partition end offset: 1
2025-11-30T08:31:13.118801Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-11-30T08:31:13.118812Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.118818Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 3. All 1 blobs are from cache.
2025-11-30T08:31:13.118821Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.118835Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 1 blobs
2025-11-30T08:31:13.118946Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 1 count 1 size 262182 from pos 0 cbcount 1
2025-11-30T08:31:13.118968Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1022: Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp done, result 1764491473112 queuesize 0 startOffset 0
2025-11-30T08:31:13.118885Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.119313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524450:3392] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 0
    Offset: 0
    WriteTimestampMS: 1764491473112
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.121607Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.122221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524455:3395], now have 1 active actors on pipe
2025-11-30T08:31:13.122331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.122342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.122382Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|54df14b9-8483b6da-75b39639-529ed11c_1 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.122399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.122403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.122409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.122411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.122415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.122426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.122429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.122432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.122450Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.122468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.122595Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|54df14b9-8483b6da-75b39639-529ed11c_1"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.123003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524455:3395] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|54df14b9-8483b6da-75b39639-529ed11c_1"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 1
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.130950Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.131588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524458:3396], now have 1 active actors on pipe
2025-11-30T08:31:13.132103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.132121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.132151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 1 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.132167Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.132185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.132190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.132203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.132206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.132212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.133343Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 0
2025-11-30T08:31:13.133383Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 1 partNo 0 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 262214 count 1 nextOffset 2 batches 1
2025-11-30T08:31:13.133395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.133398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.133402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.133517Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 1,1 HeadOffset 1 endOffset 1 curOffset 2 d0000000000_00000000000000000001_00000_0000000001_00000? size 262204 WTime 1764491473133
2025-11-30T08:31:13.133578Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.133592Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 1 partNo 0 count 1 size 262204
2025-11-30T08:31:13.137525Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 1 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.137550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.137573Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.137583Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.137595Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 1, partNo: 0, Offset: 1 is stored on disk
2025-11-30T08:31:13.137639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.137643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.137645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.137648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.137651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.137660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.137674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.137816Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.137942Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 1
    Offset: 1
    WriteTimestampMS: 1764491473131
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.138607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524458:3396] destroyed
2025-11-30T08:31:13.139314Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.140011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524462:3398], now have 1 active actors on pipe
2025-11-30T08:31:13.140028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.140035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.140067Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b31b0725-517eaaa1-d62d17f4-60d2d9cb_2 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.140077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.140083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.140088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.140090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.140093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.140104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.140107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.140110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.140123Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.140137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.140277Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b31b0725-517eaaa1-d62d17f4-60d2d9cb_2"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.140819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524462:3398] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b31b0725-517eaaa1-d62d17f4-60d2d9cb_2"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 2
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.143144Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.144106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524465:3399], now have 1 active actors on pipe
2025-11-30T08:31:13.144209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.144222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.144243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 2 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.144254Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.144266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.144271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.144281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.144283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.144288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.145737Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 0
2025-11-30T08:31:13.145794Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 2 partNo 0 FormedBlobsCount 0 NewHead: Offset 2 PartNo 0 PackedSize 262214 count 1 nextOffset 3 batches 1
2025-11-30T08:31:13.145804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.145808Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.145811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.145920Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 2,1 HeadOffset 2 endOffset 2 curOffset 3 d0000000000_00000000000000000002_00000_0000000001_00000? size 262204 WTime 1764491473145
2025-11-30T08:31:13.145976Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.145990Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 2 partNo 0 count 1 size 262204
2025-11-30T08:31:13.148319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 2 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.148341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.148361Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.148374Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.148384Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 2, partNo: 0, Offset: 2 is stored on disk
2025-11-30T08:31:13.148414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.148418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.148420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.148423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.148427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.148436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.148447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.148552Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.148637Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.149068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524465:3399] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 2
    Offset: 2
    WriteTimestampMS: 1764491473144
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.150035Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.150545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524469:3401], now have 1 active actors on pipe
2025-11-30T08:31:13.150658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.150669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.150698Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|8ab3426e-efa50ab4-14cc2537-f5e07597_3 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.150711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.150718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.150722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.150725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.150728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.150737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.150739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.150742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.150756Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.150774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.150869Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.151160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524469:3401] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|8ab3426e-efa50ab4-14cc2537-f5e07597_3"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|8ab3426e-efa50ab4-14cc2537-f5e07597_3"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 3
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.153359Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.153841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524472:3402], now have 1 active actors on pipe
2025-11-30T08:31:13.154134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.154150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.154168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 3 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.154179Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.154191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.154199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.154207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.154209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.154213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.155328Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 0
2025-11-30T08:31:13.155372Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 3 partNo 0 FormedBlobsCount 0 NewHead: Offset 3 PartNo 0 PackedSize 262214 count 1 nextOffset 4 batches 1
2025-11-30T08:31:13.155381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.155388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.155391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.155487Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 3,1 HeadOffset 3 endOffset 3 curOffset 4 d0000000000_00000000000000000003_00000_0000000001_00000? size 262204 WTime 1764491473155
2025-11-30T08:31:13.155541Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.155558Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 3 partNo 0 count 1 size 262204
2025-11-30T08:31:13.157971Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 3 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.157991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.158010Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.158018Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.158031Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 3, partNo: 0, Offset: 3 is stored on disk
2025-11-30T08:31:13.158042Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.158061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.158064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.158067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.158069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.158071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.158077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.158087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.158184Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 3
    Offset: 3
    WriteTimestampMS: 1764491473153
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 1
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-30T08:31:13.158772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524472:3402] destroyed
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.159355Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.159807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524476:3404], now have 1 active actors on pipe
2025-11-30T08:31:13.159923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.159935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.159963Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|4ef78df0-cd643d52-b486ea6c-ef52ca12_4 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.159974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.159977Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.159981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.159983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.159986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.159993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.159996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.159999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.160009Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.160019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.160073Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|4ef78df0-cd643d52-b486ea6c-ef52ca12_4"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2025-11-30T08:31:13.160301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524476:3404] destroyed
30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|4ef78df0-cd643d52-b486ea6c-ef52ca12_4"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 4
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.162509Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.163065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524479:3405], now have 1 active actors on pipe
2025-11-30T08:31:13.163403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.163415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.163434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 4 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.163443Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.163454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.163458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.163466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.163468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.163472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.164695Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 0
2025-11-30T08:31:13.164729Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 4 partNo 0 FormedBlobsCount 0 NewHead: Offset 4 PartNo 0 PackedSize 262214 count 1 nextOffset 5 batches 1
2025-11-30T08:31:13.164737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.164740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.164743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.164860Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 4,1 HeadOffset 4 endOffset 4 curOffset 5 d0000000000_00000000000000000004_00000_0000000001_00000? size 262204 WTime 1764491473156
2025-11-30T08:31:13.164907Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.164919Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 4 partNo 0 count 1 size 262204
2025-11-30T08:31:13.167507Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 4 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.167532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.167579Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.169433Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:492: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-11-30T08:31:13.169469Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.169478Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.169490Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 4, partNo: 0, Offset: 4 is stored on disk
2025-11-30T08:31:13.169526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.169537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.169542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.169546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.169548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.169558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.169573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.170078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524479:3405] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 4
    Offset: 4
    WriteTimestampMS: 1764491473156
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 13
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.169694Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.171295Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.171922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524484:3408], now have 1 active actors on pipe
2025-11-30T08:31:13.171938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.171944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.171971Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|98433dc0-6c1141b2-ae252a68-9baae532_5 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.171980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.171983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.171988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.171991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.171995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.172004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.172007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.172009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.172023Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.172035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.172181Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|98433dc0-6c1141b2-ae252a68-9baae532_5"
    Status: Active
    SeqNo: 0
  }
}
2025-11-30T08:31:13.172419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524484:3408] destroyed
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|98433dc0-6c1141b2-ae252a68-9baae532_5"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 5
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.174653Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.175169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524487:3409], now have 1 active actors on pipe
2025-11-30T08:31:13.175525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.175536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.175556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 5 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.175565Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.175576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.175579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.175587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.175590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.175594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.176810Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 0
2025-11-30T08:31:13.176851Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 5 partNo 0 FormedBlobsCount 0 NewHead: Offset 5 PartNo 0 PackedSize 262214 count 1 nextOffset 6 batches 1
2025-11-30T08:31:13.176859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.176862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.176865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.176962Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 5,1 HeadOffset 5 endOffset 5 curOffset 6 d0000000000_00000000000000000005_00000_0000000001_00000? size 262204 WTime 1764491473176
2025-11-30T08:31:13.177006Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.177019Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 5 partNo 0 count 1 size 262204
2025-11-30T08:31:13.179330Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 5 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.179348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.179365Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.179372Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.179381Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 5, partNo: 0, Offset: 5 is stored on disk
2025-11-30T08:31:13.179410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.179413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.179415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.179417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.179419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.179425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.179434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.179546Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.179611Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.179944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524487:3409] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 5
    Offset: 5
    WriteTimestampMS: 1764491473175
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.181464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524491:3411], now have 1 active actors on pipe
2025-11-30T08:31:13.181494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.181501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.181529Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|bb046933-ae8dd28d-7642fcd5-7e7a4f35_6 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.181535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.181538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.181542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.181544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.181547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.181554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.181556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.181558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.181568Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.181578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
2025-11-30T08:31:13.181973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524491:3411] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|bb046933-ae8dd28d-7642fcd5-7e7a4f35_6"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|bb046933-ae8dd28d-7642fcd5-7e7a4f35_6"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 6
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.184476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524494:3412], now have 1 active actors on pipe
2025-11-30T08:31:13.180700Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.181712Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.184055Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.184836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.184846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.184864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 6 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.184872Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.184884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.184887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.184894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.184896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.184900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.186006Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 0
2025-11-30T08:31:13.186036Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 6 partNo 0 FormedBlobsCount 0 NewHead: Offset 6 PartNo 0 PackedSize 262214 count 1 nextOffset 7 batches 1
2025-11-30T08:31:13.186042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.186045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.186048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.186135Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 6,1 HeadOffset 6 endOffset 6 curOffset 7 d0000000000_00000000000000000006_00000_0000000001_00000? size 262204 WTime 1764491473186
2025-11-30T08:31:13.186175Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.186187Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 6 partNo 0 count 1 size 262204
2025-11-30T08:31:13.190456Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 6 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.190474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.190492Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.190500Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.190509Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 6, partNo: 0, Offset: 6 is stored on disk
2025-11-30T08:31:13.190536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.190541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.190543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.190546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.190548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.190554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.190563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.190674Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.190749Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-30T08:31:13.191110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524494:3412] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 6
    Offset: 6
    WriteTimestampMS: 1764491473184
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.191706Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.192262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524498:3414], now have 1 active actors on pipe
2025-11-30T08:31:13.192274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.192279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.192304Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|3e6907f-b0ab7640-550aff11-a5ac6ce9_7 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.192311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.192314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.192318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.192320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.192323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.192330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.192333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.192337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.192347Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.192357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.192501Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|3e6907f-b0ab7640-550aff11-a5ac6ce9_7"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.192840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524498:3414] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|3e6907f-b0ab7640-550aff11-a5ac6ce9_7"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 7
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.194835Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.195619Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524501:3415], now have 1 active actors on pipe
2025-11-30T08:31:13.195695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.195701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.195719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 7 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.195726Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.195735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.195739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.195747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.195749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.195753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.196896Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 0
2025-11-30T08:31:13.196923Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 7 partNo 0 FormedBlobsCount 0 NewHead: Offset 7 PartNo 0 PackedSize 262214 count 1 nextOffset 8 batches 1
2025-11-30T08:31:13.196931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.196934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.196938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.197039Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 7,1 HeadOffset 7 endOffset 7 curOffset 8 d0000000000_00000000000000000007_00000_0000000001_00000? size 262204 WTime 1764491473196
2025-11-30T08:31:13.197084Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.197097Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 7 partNo 0 count 1 size 262204
2025-11-30T08:31:13.200054Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 7 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.200072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.200095Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.200103Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.200112Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 7, partNo: 0, Offset: 7 is stored on disk
2025-11-30T08:31:13.200146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.200149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.200151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.200154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.200156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.200163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.200173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.200280Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.200378Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 7
    Offset: 7
    WriteTimestampMS: 1764491473195
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.200873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524501:3415] destroyed
2025-11-30T08:31:13.201282Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.201427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.201437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.201444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.201447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.201449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.201659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524505:3417], now have 1 active actors on pipe
2025-11-30T08:31:13.201842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.201849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.201877Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|fca8cf1a-5b1fb828-2f8fc52-262c9cfb_8 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.201885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.201889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.201893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.201895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.201898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.201906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.201909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.201912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.201922Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.201933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.202013Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|fca8cf1a-5b1fb828-2f8fc52-262c9cfb_8"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.202346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524505:3417] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|fca8cf1a-5b1fb828-2f8fc52-262c9cfb_8"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 8
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.204184Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.204563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524508:3418], now have 1 active actors on pipe
2025-11-30T08:31:13.204914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.204933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.204954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 8 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.204963Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.204974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.204978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.204986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.204988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.204992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.205968Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 0
2025-11-30T08:31:13.206011Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 8 partNo 0 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 262214 count 1 nextOffset 9 batches 1
2025-11-30T08:31:13.206020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.206023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.206026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.206124Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 8,1 HeadOffset 8 endOffset 8 curOffset 9 d0000000000_00000000000000000008_00000_0000000001_00000? size 262204 WTime 1764491473206
2025-11-30T08:31:13.206185Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.206205Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 8 partNo 0 count 1 size 262204
2025-11-30T08:31:13.209751Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 8 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.209786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.209803Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.209814Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.209826Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.209838Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 8, partNo: 0, Offset: 8 is stored on disk
2025-11-30T08:31:13.209883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.209886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.209888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.209890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.209892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.209902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.209916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.210010Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 8
    Offset: 8
    WriteTimestampMS: 1764491473204
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.210993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524508:3418] destroyed
2025-11-30T08:31:13.211195Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.211708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524515:3422], now have 1 active actors on pipe
2025-11-30T08:31:13.211861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.211877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.211909Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a84d8c3d-50740393-9778070f-84bdba91_9 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.211924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.211929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.211935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.211937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.211941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.211951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.211954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.211957Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.211971Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.211984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.212154Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a84d8c3d-50740393-9778070f-84bdba91_9"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a84d8c3d-50740393-9778070f-84bdba91_9"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 9
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.213662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524515:3422] destroyed
2025-11-30T08:31:13.215224Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.215815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524518:3423], now have 1 active actors on pipe
2025-11-30T08:31:13.216381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.216397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.216422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 9 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.216437Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.216458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.216463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.216478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.216481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.216487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.217574Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 0
2025-11-30T08:31:13.217635Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 9 partNo 0 FormedBlobsCount 0 NewHead: Offset 9 PartNo 0 PackedSize 262214 count 1 nextOffset 10 batches 1
2025-11-30T08:31:13.217648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.217652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.217656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.217780Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 9,1 HeadOffset 9 endOffset 9 curOffset 10 d0000000000_00000000000000000009_00000_0000000001_00000? size 262204 WTime 1764491473217
2025-11-30T08:31:13.217839Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.217853Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 9 partNo 0 count 1 size 262204
2025-11-30T08:31:13.220360Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 9 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.220393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.220411Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.220424Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.220442Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.220458Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 9, partNo: 0, Offset: 9 is stored on disk
2025-11-30T08:31:13.220505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.220513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.220516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.220518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.220522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.220533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.220552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.220650Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 9
    Offset: 9
    WriteTimestampMS: 1764491473216
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.221261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524518:3423] destroyed
2025-11-30T08:31:13.221764Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.222270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524522:3425], now have 1 active actors on pipe
2025-11-30T08:31:13.222351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.222366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.222409Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|3f295f2-86638a88-711e034c-f51f6e6_10 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.222426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.222431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.222437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.222440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.222444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.222454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.222458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.222461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.222478Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.222501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.222565Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|3f295f2-86638a88-711e034c-f51f6e6_10"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|3f295f2-86638a88-711e034c-f51f6e6_10"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 10
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.223916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524522:3425] destroyed
2025-11-30T08:31:13.225033Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.225570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524525:3426], now have 1 active actors on pipe
2025-11-30T08:31:13.226063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.226070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.226093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 10 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.226105Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.226121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.226125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.226136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.226138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.226142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.226843Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 0
2025-11-30T08:31:13.226875Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 10 partNo 0 FormedBlobsCount 0 NewHead: Offset 10 PartNo 0 PackedSize 262214 count 1 nextOffset 11 batches 1
2025-11-30T08:31:13.226884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.226887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.226890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.226983Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 10,1 HeadOffset 10 endOffset 10 curOffset 11 d0000000000_00000000000000000010_00000_0000000001_00000? size 262204 WTime 1764491473226
2025-11-30T08:31:13.227027Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.227036Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 10 partNo 0 count 1 size 262204
2025-11-30T08:31:13.231344Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 10 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.231367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.231390Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.231399Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.231408Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 10, partNo: 0, Offset: 10 is stored on disk
2025-11-30T08:31:13.231439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.231441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.231442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.231444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.231445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.231453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.231463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.231500Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.231621Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 10
    Offset: 10
    WriteTimestampMS: 1764491473225
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.232157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524525:3426] destroyed
2025-11-30T08:31:13.232752Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.233216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524529:3428], now have 1 active actors on pipe
2025-11-30T08:31:13.233337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.233346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.233376Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|be0f314d-74037b44-bf4b5500-c82a756a_11 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.233386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.233390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.233395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.233397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.233400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.233410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.233413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.233415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.233431Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.233445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.233597Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|be0f314d-74037b44-bf4b5500-c82a756a_11"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.234011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524529:3428] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|be0f314d-74037b44-bf4b5500-c82a756a_11"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 11
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.236276Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.236737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524532:3429], now have 1 active actors on pipe
2025-11-30T08:31:13.237237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.237250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.237279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 11 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.237292Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.237310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.237316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.237327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.237329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.237333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.238286Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 0
2025-11-30T08:31:13.238325Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 11 partNo 0 FormedBlobsCount 0 NewHead: Offset 11 PartNo 0 PackedSize 262214 count 1 nextOffset 12 batches 1
2025-11-30T08:31:13.238335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.238338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.238341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.238447Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 11,1 HeadOffset 11 endOffset 11 curOffset 12 d0000000000_00000000000000000011_00000_0000000001_00000? size 262204 WTime 1764491473238
2025-11-30T08:31:13.238501Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.238514Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 11 partNo 0 count 1 size 262204
2025-11-30T08:31:13.241609Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 11 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.241635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.241661Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.241674Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.241688Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 11, partNo: 0, Offset: 11 is stored on disk
2025-11-30T08:31:13.241733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.241736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.241738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.241742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.241745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.241756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.241771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.241906Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.242032Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 11
    Offset: 11
    WriteTimestampMS: 1764491473237
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.242749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524532:3429] destroyed
2025-11-30T08:31:13.243260Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.243877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524536:3431], now have 1 active actors on pipe
2025-11-30T08:31:13.243895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.243902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.243936Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|fd6d03f1-bcb71e20-8bffc740-f06bdb2f_12 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.243946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.243950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.243956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.243958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.243962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.243973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.243976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.243978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.243997Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.244014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.244168Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|fd6d03f1-bcb71e20-8bffc740-f06bdb2f_12"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.244745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524536:3431] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|fd6d03f1-bcb71e20-8bffc740-f06bdb2f_12"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 12
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.247149Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.248013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524539:3432], now have 1 active actors on pipe
2025-11-30T08:31:13.248116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.248124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.248148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 12 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.248160Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.248179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.248184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.248197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.248199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.248204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.249172Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 0
2025-11-30T08:31:13.249214Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 12 partNo 0 FormedBlobsCount 0 NewHead: Offset 12 PartNo 0 PackedSize 262214 count 1 nextOffset 13 batches 1
2025-11-30T08:31:13.249225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.249229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.249233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.249338Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 12,1 HeadOffset 12 endOffset 12 curOffset 13 d0000000000_00000000000000000012_00000_0000000001_00000? size 262204 WTime 1764491473249
2025-11-30T08:31:13.249390Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.249403Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 12 partNo 0 count 1 size 262204
2025-11-30T08:31:13.252215Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 12 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.252240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.252267Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.252279Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.252294Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 12, partNo: 0, Offset: 12 is stored on disk
2025-11-30T08:31:13.252334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.252338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.252340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.252343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.252346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.252356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.252370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.252507Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.252595Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 12
    Offset: 12
    WriteTimestampMS: 1764491473247
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-30T08:31:13.253241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524539:3432] destroyed
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.253924Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.254491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524543:3434], now have 1 active actors on pipe
2025-11-30T08:31:13.254576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.254587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.254623Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f1441752-35993b58-9a9ee22c-3cddf903_13 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.254634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.254639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.254645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.254647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.254651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.254663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.254666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.254669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.254689Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.254706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.255313Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f1441752-35993b58-9a9ee22c-3cddf903_13"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f1441752-35993b58-9a9ee22c-3cddf903_13"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 13
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.257460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524543:3434] destroyed
2025-11-30T08:31:13.257917Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.258399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524546:3435], now have 1 active actors on pipe
2025-11-30T08:31:13.258888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.258903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.258929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 13 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.258941Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.258959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.258964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.258977Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.258979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.258984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.259900Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 0
2025-11-30T08:31:13.259942Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 13 partNo 0 FormedBlobsCount 0 NewHead: Offset 13 PartNo 0 PackedSize 262214 count 1 nextOffset 14 batches 1
2025-11-30T08:31:13.259952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.259956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.259960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.260063Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 13,1 HeadOffset 13 endOffset 13 curOffset 14 d0000000000_00000000000000000013_00000_0000000001_00000? size 262204 WTime 1764491473259
2025-11-30T08:31:13.260113Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.260127Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 13 partNo 0 count 1 size 262204
2025-11-30T08:31:13.262484Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 13 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.262508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.262533Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.262545Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.262555Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 13, partNo: 0, Offset: 13 is stored on disk
2025-11-30T08:31:13.262593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.262595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.262598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.262600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.262602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.262612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.262626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.262633Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.262763Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.263155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524546:3435] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 13
    Offset: 13
    WriteTimestampMS: 1764491473258
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.263979Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.264419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524550:3437], now have 1 active actors on pipe
2025-11-30T08:31:13.264516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.264526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.264563Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b89b8816-99826b61-7634ac01-d7222e1d_14 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.264572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.264575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.264581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.264583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.264588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.264598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.264601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.264605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.264620Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.264635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.264693Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b89b8816-99826b61-7634ac01-d7222e1d_14"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b89b8816-99826b61-7634ac01-d7222e1d_14"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 14
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.266166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524550:3437] destroyed
2025-11-30T08:31:13.268469Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.270379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524553:3438], now have 1 active actors on pipe
2025-11-30T08:31:13.270950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.270962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.270986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 14 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.270999Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.271015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.271019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.271031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.271033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.271038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.272066Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 0
2025-11-30T08:31:13.272108Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 14 partNo 0 FormedBlobsCount 0 NewHead: Offset 14 PartNo 0 PackedSize 262214 count 1 nextOffset 15 batches 1
2025-11-30T08:31:13.272117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.272120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.272126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.272244Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 14,1 HeadOffset 14 endOffset 14 curOffset 15 d0000000000_00000000000000000014_00000_0000000001_00000? size 262204 WTime 1764491473272
2025-11-30T08:31:13.272298Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.272310Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 14 partNo 0 count 1 size 262204
2025-11-30T08:31:13.274699Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 14 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.274723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.274739Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.274746Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.274757Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.274768Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 14, partNo: 0, Offset: 14 is stored on disk
2025-11-30T08:31:13.274800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.274803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.274804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.274807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.274810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.274820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.274833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.274915Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 14
    Offset: 14
    WriteTimestampMS: 1764491473271
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.275711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524553:3438] destroyed
2025-11-30T08:31:13.276022Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.276487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524557:3440], now have 1 active actors on pipe
2025-11-30T08:31:13.276628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.276638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.276674Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|54074939-d1cb1dd6-eafd781a-38782aa1_15 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.276684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.276687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.276693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.276694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.276697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.276707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.276710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.276713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.276728Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.276743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.276825Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|54074939-d1cb1dd6-eafd781a-38782aa1_15"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.277684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524557:3440] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|54074939-d1cb1dd6-eafd781a-38782aa1_15"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 15
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.279396Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.279785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524560:3441], now have 1 active actors on pipe
2025-11-30T08:31:13.280248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.280263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.280288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 15 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.280307Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.280322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.280330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.280342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.280347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.280351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.281499Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 0
2025-11-30T08:31:13.281542Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 15 partNo 0 FormedBlobsCount 0 NewHead: Offset 15 PartNo 0 PackedSize 262214 count 1 nextOffset 16 batches 1
2025-11-30T08:31:13.281551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.281560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.281563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.281676Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 15,1 HeadOffset 15 endOffset 15 curOffset 16 d0000000000_00000000000000000015_00000_0000000001_00000? size 262204 WTime 1764491473277
2025-11-30T08:31:13.281738Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.281760Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 15 partNo 0 count 1 size 262204
2025-11-30T08:31:13.284269Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 15 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.284296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.284317Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.284323Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.284334Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.284345Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 15, partNo: 0, Offset: 15 is stored on disk
2025-11-30T08:31:13.284391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.284400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.284403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.284405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.284408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.284418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.284432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.284566Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 15
    Offset: 15
    WriteTimestampMS: 1764491473277
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 6
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.285780Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.284964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524560:3441] destroyed
2025-11-30T08:31:13.286200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524565:3444], now have 1 active actors on pipe
2025-11-30T08:31:13.286299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.286309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.286344Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|2e563296-51a32d44-5772b53d-f316641_16 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.286353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.286356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.286361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.286363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.286367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.286376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.286381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.286384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.286398Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.286411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
2025-11-30T08:31:13.286731Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|2e563296-51a32d44-5772b53d-f316641_16"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|2e563296-51a32d44-5772b53d-f316641_16"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 16
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.288236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524565:3444] destroyed
2025-11-30T08:31:13.289207Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.289745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524568:3445], now have 1 active actors on pipe
2025-11-30T08:31:13.290179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.290192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.290217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 16 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.290228Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.290244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.290248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.290259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.290262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.290266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.291521Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 0
2025-11-30T08:31:13.291578Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 16 partNo 0 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 262214 count 1 nextOffset 17 batches 1
2025-11-30T08:31:13.291589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.291594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.291597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.291705Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 16,1 HeadOffset 16 endOffset 16 curOffset 17 d0000000000_00000000000000000016_00000_0000000001_00000? size 262204 WTime 1764491473291
2025-11-30T08:31:13.291759Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.291771Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 16 partNo 0 count 1 size 262204
2025-11-30T08:31:13.294384Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 16 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.294410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.294431Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.294434Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.294445Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.294458Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 16, partNo: 0, Offset: 16 is stored on disk
2025-11-30T08:31:13.294494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.294496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.294498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.294501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.294503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.294512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.294525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.294672Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 16
    Offset: 16
    WriteTimestampMS: 1764491473289
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.295617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524568:3445] destroyed
2025-11-30T08:31:13.297318Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.297851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524572:3447], now have 1 active actors on pipe
2025-11-30T08:31:13.297943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.297954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.297993Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|eef16ab9-e673b4df-c769a64a-e4b86eb2_17 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.298005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.298009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.298015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.298017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.298021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.298031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.298034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.298037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.298055Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.298072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.298480Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|eef16ab9-e673b4df-c769a64a-e4b86eb2_17"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|eef16ab9-e673b4df-c769a64a-e4b86eb2_17"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 17
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.298939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524572:3447] destroyed
2025-11-30T08:31:13.301515Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.301789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.301802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.301805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.301809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.301812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.301910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524575:3448], now have 1 active actors on pipe
2025-11-30T08:31:13.302267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.302303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.302329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 17 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.302341Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.302354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.302358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.302369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.302371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.302376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.303646Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 0
2025-11-30T08:31:13.303684Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 17 partNo 0 FormedBlobsCount 0 NewHead: Offset 17 PartNo 0 PackedSize 262214 count 1 nextOffset 18 batches 1
2025-11-30T08:31:13.303695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.303698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.303702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.303812Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 17,1 HeadOffset 17 endOffset 17 curOffset 18 d0000000000_00000000000000000017_00000_0000000001_00000? size 262204 WTime 1764491473303
2025-11-30T08:31:13.303887Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.303900Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 17 partNo 0 count 1 size 262204
2025-11-30T08:31:13.306429Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 17 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.306456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.306477Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.306480Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.306491Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.306502Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 17, partNo: 0, Offset: 17 is stored on disk
2025-11-30T08:31:13.306541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.306549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.306551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.306554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.306556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.306565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.306578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.306719Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 17
    Offset: 17
    WriteTimestampMS: 1764491473302
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.307177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524575:3448] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.307839Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.308329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524579:3450], now have 1 active actors on pipe
2025-11-30T08:31:13.308434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.308446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.308483Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|bbbfb8bb-58e46838-a443d10e-c7b6ea87_18 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.308498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.308501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.308506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.308508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.308511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.308521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.308524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.308528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.308543Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.308557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.308703Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|bbbfb8bb-58e46838-a443d10e-c7b6ea87_18"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.309222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524579:3450] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|bbbfb8bb-58e46838-a443d10e-c7b6ea87_18"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 18
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.310683Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.311149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524582:3451], now have 1 active actors on pipe
2025-11-30T08:31:13.311462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.311474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.311496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 18 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.311507Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.311519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.311524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.311535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.311537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.311540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.312749Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 0
2025-11-30T08:31:13.312808Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 18 partNo 0 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 262214 count 1 nextOffset 19 batches 1
2025-11-30T08:31:13.312818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.312821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.312824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.312933Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 18,1 HeadOffset 18 endOffset 18 curOffset 19 d0000000000_00000000000000000018_00000_0000000001_00000? size 262204 WTime 1764491473312
2025-11-30T08:31:13.312985Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.312998Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 18 partNo 0 count 1 size 262204
2025-11-30T08:31:13.315601Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 18 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.315622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.315644Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.315653Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.315664Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 18, partNo: 0, Offset: 18 is stored on disk
2025-11-30T08:31:13.315701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.315703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.315706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.315709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.315710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.315718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.315730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.315868Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.315981Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 18
    Offset: 18
    WriteTimestampMS: 1764491473311
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-30T08:31:13.316400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524582:3451] destroyed
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.317131Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.317601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524586:3453], now have 1 active actors on pipe
2025-11-30T08:31:13.317730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.317738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.317768Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|7a118451-3f788b4a-e240e1de-b1ba0b7f_19 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.317775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.317779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.317783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.317785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.317788Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.317797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.317801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.317804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.317820Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.317832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.317976Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|7a118451-3f788b4a-e240e1de-b1ba0b7f_19"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.318285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524586:3453] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|7a118451-3f788b4a-e240e1de-b1ba0b7f_19"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 19
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.320371Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.320843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524589:3454], now have 1 active actors on pipe
2025-11-30T08:31:13.321332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.321345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.321369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 19 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.321379Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.321391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.321398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.321408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.321409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.321413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.322518Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 0
2025-11-30T08:31:13.322548Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 19 partNo 0 FormedBlobsCount 0 NewHead: Offset 19 PartNo 0 PackedSize 262214 count 1 nextOffset 20 batches 1
2025-11-30T08:31:13.322558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.322561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.322564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.322668Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 19,1 HeadOffset 19 endOffset 19 curOffset 20 d0000000000_00000000000000000019_00000_0000000001_00000? size 262204 WTime 1764491473322
2025-11-30T08:31:13.322721Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.322732Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 19 partNo 0 count 1 size 262204
2025-11-30T08:31:13.325299Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 19 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.325320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.325342Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.325352Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.325363Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 19, partNo: 0, Offset: 19 is stored on disk
2025-11-30T08:31:13.325402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.325406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.325408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.325410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.325412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.325421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.325433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.325489Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.325580Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.326135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524589:3454] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 19
    Offset: 19
    WriteTimestampMS: 1764491473321
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.337690Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.338533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524594:3457], now have 1 active actors on pipe
2025-11-30T08:31:13.338675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.338691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.338740Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|cd6955ed-f5a600ac-2d1fdc71-d08743d1_20 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.338759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.338766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.338773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.338776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.338781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.338794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.338798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.338802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.338822Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.338844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.338989Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.339478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524594:3457] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|cd6955ed-f5a600ac-2d1fdc71-d08743d1_20"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|cd6955ed-f5a600ac-2d1fdc71-d08743d1_20"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 20
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.343487Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.344846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524597:3458], now have 1 active actors on pipe
2025-11-30T08:31:13.344996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.345013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.345041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 20 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.345061Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.345084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.345094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.345107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.345110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.345115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.346236Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 0
2025-11-30T08:31:13.346292Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 20 partNo 0 FormedBlobsCount 0 NewHead: Offset 20 PartNo 0 PackedSize 262214 count 1 nextOffset 21 batches 1
2025-11-30T08:31:13.346305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.346313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.346317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.346428Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 20,1 HeadOffset 20 endOffset 20 curOffset 21 d0000000000_00000000000000000020_00000_0000000001_00000? size 262204 WTime 1764491473346
2025-11-30T08:31:13.346486Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.346504Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 20 partNo 0 count 1 size 262204
2025-11-30T08:31:13.358104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 20 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.358129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.358155Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.358166Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.358177Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 20, partNo: 0, Offset: 20 is stored on disk
2025-11-30T08:31:13.358217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.358222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.358225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.358228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.358230Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.358238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.358250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.358378Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 0 suffix '63' size 262204
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 20
    Offset: 20
    WriteTimestampMS: 1764491473344
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 11
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.358504Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.359758Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.366151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524597:3458] destroyed
2025-11-30T08:31:13.366172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524601:3460], now have 1 active actors on pipe
2025-11-30T08:31:13.366195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.366202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.366244Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|c12aed22-30d091ee-bfdb225f-974296e4_21 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.366254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.366258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.366263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.366266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.366270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.366279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.366282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.366289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.366305Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.366319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|c12aed22-30d091ee-bfdb225f-974296e4_21"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|c12aed22-30d091ee-bfdb225f-974296e4_21"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 21
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.366443Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.368959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524601:3460] destroyed
2025-11-30T08:31:13.369951Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.370521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524604:3461], now have 1 active actors on pipe
2025-11-30T08:31:13.370969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.370977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.370997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 21 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.371007Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.371019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.371023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.371031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.371033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.371040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.372229Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 0
2025-11-30T08:31:13.372263Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 21 partNo 0 FormedBlobsCount 0 NewHead: Offset 21 PartNo 0 PackedSize 262214 count 1 nextOffset 22 batches 1
2025-11-30T08:31:13.372270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.372273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.372278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.372367Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 21,1 HeadOffset 21 endOffset 21 curOffset 22 d0000000000_00000000000000000021_00000_0000000001_00000? size 262204 WTime 1764491473371
2025-11-30T08:31:13.372417Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.372429Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 21 partNo 0 count 1 size 262204
2025-11-30T08:31:13.382977Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.382488Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 21 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.382518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.382545Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.382557Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.382577Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 21, partNo: 0, Offset: 21 is stored on disk
2025-11-30T08:31:13.382624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.382626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.382628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.382630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.382631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.382642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.382654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.382900Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.383357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524604:3461] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 21
    Offset: 21
    WriteTimestampMS: 1764491473370
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 10
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.388365Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.390906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524609:3464], now have 1 active actors on pipe
2025-11-30T08:31:13.391178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.391193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.391235Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|2de2e72a-b20190f0-73dd4886-18c09f75_22 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.391248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.391253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.391259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.391261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.391271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.391284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.391288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.391291Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.391312Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.391335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.391435Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.391844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524609:3464] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|2de2e72a-b20190f0-73dd4886-18c09f75_22"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|2de2e72a-b20190f0-73dd4886-18c09f75_22"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 22
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.395687Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.396561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524612:3465], now have 1 active actors on pipe
2025-11-30T08:31:13.397084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.397094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.397116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 22 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.397128Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.397145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.397149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.397159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.397161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.397165Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.397994Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 0
2025-11-30T08:31:13.398043Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 22 partNo 0 FormedBlobsCount 0 NewHead: Offset 22 PartNo 0 PackedSize 262214 count 1 nextOffset 23 batches 1
2025-11-30T08:31:13.398051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.398054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.398056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.398157Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 22,1 HeadOffset 22 endOffset 22 curOffset 23 d0000000000_00000000000000000022_00000_0000000001_00000? size 262204 WTime 1764491473397
2025-11-30T08:31:13.398210Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.398222Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 22 partNo 0 count 1 size 262204
2025-11-30T08:31:13.400698Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 22 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.400721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.400747Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.400765Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.400776Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 22, partNo: 0, Offset: 22 is stored on disk
2025-11-30T08:31:13.400847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.400854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.400856Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.400858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.400860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.400870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.400885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.400988Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.401102Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 22
    Offset: 22
    WriteTimestampMS: 1764491473396
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.401573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524612:3465] destroyed
2025-11-30T08:31:13.402118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.402125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.402127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.402130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.402131Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.402338Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.402938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524616:3467], now have 1 active actors on pipe
2025-11-30T08:31:13.402955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.402962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.402992Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|661830f5-b8cae545-716af7d4-1efe35db_23 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.403001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.403004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.403009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.403011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.403014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.403023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.403026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.403028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.403040Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.403064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.403281Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|661830f5-b8cae545-716af7d4-1efe35db_23"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.403593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524616:3467] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|661830f5-b8cae545-716af7d4-1efe35db_23"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 23
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.406014Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.406740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524619:3468], now have 1 active actors on pipe
2025-11-30T08:31:13.406851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.406864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.406889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 23 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.406906Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.406921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.406926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.406939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.406945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.406950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.407845Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 0
2025-11-30T08:31:13.407899Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 23 partNo 0 FormedBlobsCount 0 NewHead: Offset 23 PartNo 0 PackedSize 262214 count 1 nextOffset 24 batches 1
2025-11-30T08:31:13.407912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.407922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.407925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.408025Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 23,1 HeadOffset 23 endOffset 23 curOffset 24 d0000000000_00000000000000000023_00000_0000000001_00000? size 262204 WTime 1764491473407
2025-11-30T08:31:13.408072Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.408081Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 23 partNo 0 count 1 size 262204
2025-11-30T08:31:13.411149Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 23 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.411175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.411198Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.411212Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.411228Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 23, partNo: 0, Offset: 23 is stored on disk
2025-11-30T08:31:13.411275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.411281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.411283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.411287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.411289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.411298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.411312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.411371Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.411438Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 23
    Offset: 23
    WriteTimestampMS: 1764491473406
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.412306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524619:3468] destroyed
2025-11-30T08:31:13.412688Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.413137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524623:3470], now have 1 active actors on pipe
2025-11-30T08:31:13.413153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.413159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.413191Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f6f99649-d1674a57-6b782c81-c16cd561_24 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.413203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.413206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.413213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.413215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.413218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.413227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.413231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.413234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.413248Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.413261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.413373Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-30T08:31:13.413709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524623:3470] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f6f99649-d1674a57-6b782c81-c16cd561_24"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f6f99649-d1674a57-6b782c81-c16cd561_24"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 24
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.415428Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.416179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524626:3471], now have 1 active actors on pipe
2025-11-30T08:31:13.416268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.416278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.416303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 24 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.416316Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.416333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.416342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.416353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.416355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.416359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.417314Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 0
2025-11-30T08:31:13.417416Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 24 partNo 0 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 262214 count 1 nextOffset 25 batches 1
2025-11-30T08:31:13.417436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.417439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.417442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.417561Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 24,1 HeadOffset 24 endOffset 24 curOffset 25 d0000000000_00000000000000000024_00000_0000000001_00000? size 262204 WTime 1764491473417
2025-11-30T08:31:13.417613Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.417626Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 24 partNo 0 count 1 size 262204
2025-11-30T08:31:13.419819Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 24 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.419839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.419873Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.419883Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.419896Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 24, partNo: 0, Offset: 24 is stored on disk
2025-11-30T08:31:13.419930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.419933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.419935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.419938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.419940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.419949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.419960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.419996Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.420140Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.420466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524626:3471] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 24
    Offset: 24
    WriteTimestampMS: 1764491473415
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 1
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.421722Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.422206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524630:3473], now have 1 active actors on pipe
2025-11-30T08:31:13.422224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.422229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.422260Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|3f412715-331c4c2c-7b67fc07-40da464c_25 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.422268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.422271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.422275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.422277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.422280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.422289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.422292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.422294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.422309Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.422321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.422439Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-30T08:31:13.422731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524630:3473] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|3f412715-331c4c2c-7b67fc07-40da464c_25"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|3f412715-331c4c2c-7b67fc07-40da464c_25"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 25
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.424908Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.425375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524633:3474], now have 1 active actors on pipe
2025-11-30T08:31:13.425817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.425830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.425854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 25 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.425865Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.425879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.425883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.425893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.425895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.425899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.426976Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 0
2025-11-30T08:31:13.427024Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 25 partNo 0 FormedBlobsCount 0 NewHead: Offset 25 PartNo 0 PackedSize 262214 count 1 nextOffset 26 batches 1
2025-11-30T08:31:13.427039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.427042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.427045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.427144Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 25,1 HeadOffset 25 endOffset 25 curOffset 26 d0000000000_00000000000000000025_00000_0000000001_00000? size 262204 WTime 1764491473426
2025-11-30T08:31:13.427203Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.427222Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 25 partNo 0 count 1 size 262204
2025-11-30T08:31:13.429911Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 25 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.429938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.429962Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.429962Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.429973Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.429984Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 25, partNo: 0, Offset: 25 is stored on disk
2025-11-30T08:31:13.430021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.430024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.430027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.430030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.430032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.430040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.430055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.430239Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 25
    Offset: 25
    WriteTimestampMS: 1764491473424
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 2
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-30T08:31:13.430701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524633:3474] destroyed
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.431353Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.431979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524637:3476], now have 1 active actors on pipe
2025-11-30T08:31:13.432013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.432020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.432058Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ae7bdf82-bdc4b433-83591ac6-76213b95_26 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.432067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.432071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.432077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.432079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.432083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.432094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.432097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.432101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.432119Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.432133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.432297Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ae7bdf82-bdc4b433-83591ac6-76213b95_26"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.432682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524637:3476] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ae7bdf82-bdc4b433-83591ac6-76213b95_26"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 26
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.434655Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.435386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524640:3477], now have 1 active actors on pipe
2025-11-30T08:31:13.435501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.435508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.435531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 26 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.435542Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.435555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.435559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.435571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.435574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.435578Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.436770Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 0
2025-11-30T08:31:13.436847Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 26 partNo 0 FormedBlobsCount 0 NewHead: Offset 26 PartNo 0 PackedSize 262214 count 1 nextOffset 27 batches 1
2025-11-30T08:31:13.436858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.436861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.436865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.436972Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 26,1 HeadOffset 26 endOffset 26 curOffset 27 d0000000000_00000000000000000026_00000_0000000001_00000? size 262204 WTime 1764491473435
2025-11-30T08:31:13.437026Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.437039Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 26 partNo 0 count 1 size 262204
2025-11-30T08:31:13.439470Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 26 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.439496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.439513Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.441010Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:492: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-11-30T08:31:13.441040Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.441050Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.441063Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 26, partNo: 0, Offset: 26 is stored on disk
2025-11-30T08:31:13.441107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.441110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.441112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.441115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.441117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.441126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.441144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.442690Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.443138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524640:3477] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 26
    Offset: 26
    WriteTimestampMS: 1764491473435
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 3
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.443955Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.444351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524644:3479], now have 1 active actors on pipe
2025-11-30T08:31:13.444482Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.444494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.444531Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9f18cbd0-730c0a5c-e1fab6bc-f53133f7_27 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.444543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.444547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.444552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.444554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.444557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.444568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.444580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.444583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.444603Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.444620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.444845Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9f18cbd0-730c0a5c-e1fab6bc-f53133f7_27"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.445206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524644:3479] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9f18cbd0-730c0a5c-e1fab6bc-f53133f7_27"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 27
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.446861Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.447783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524647:3480], now have 1 active actors on pipe
2025-11-30T08:31:13.447921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.447935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.447959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 27 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.447976Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.447995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.448004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.448015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.448017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.448021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.449030Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 0
2025-11-30T08:31:13.449076Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 27 partNo 0 FormedBlobsCount 0 NewHead: Offset 27 PartNo 0 PackedSize 262214 count 1 nextOffset 28 batches 1
2025-11-30T08:31:13.449093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.449101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.449104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.449234Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 27,1 HeadOffset 27 endOffset 27 curOffset 28 d0000000000_00000000000000000027_00000_0000000001_00000? size 262204 WTime 1764491473448
2025-11-30T08:31:13.449294Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.449312Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 27 partNo 0 count 1 size 262204
2025-11-30T08:31:13.451706Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 27 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.451734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.451783Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.452862Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:492: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-11-30T08:31:13.452898Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.452910Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.452930Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 27, partNo: 0, Offset: 27 is stored on disk
2025-11-30T08:31:13.452982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.452991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.452993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.452996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.452999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.453011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.453027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.453145Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 27
    Offset: 27
    WriteTimestampMS: 1764491473444
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 3
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.454249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524647:3480] destroyed
2025-11-30T08:31:13.454421Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.455106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524652:3483], now have 1 active actors on pipe
2025-11-30T08:31:13.455126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.455133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.455176Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|c9a18403-5b2d6a63-6f6da7b3-48b41822_28 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.455187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.455196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.455202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.455204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.455207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.455220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.455228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.455232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.455247Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.455266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.455400Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|c9a18403-5b2d6a63-6f6da7b3-48b41822_28"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|c9a18403-5b2d6a63-6f6da7b3-48b41822_28"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 28
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.457014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524652:3483] destroyed
2025-11-30T08:31:13.457899Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.458870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524655:3484], now have 1 active actors on pipe
2025-11-30T08:31:13.458974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.458988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.459018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 28 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.459033Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.459051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.459060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.459071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.459077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.459082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.460155Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 0
2025-11-30T08:31:13.460203Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 28 partNo 0 FormedBlobsCount 0 NewHead: Offset 28 PartNo 0 PackedSize 262214 count 1 nextOffset 29 batches 1
2025-11-30T08:31:13.460219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.460227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.460231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.460343Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 28,1 HeadOffset 28 endOffset 28 curOffset 29 d0000000000_00000000000000000028_00000_0000000001_00000? size 262204 WTime 1764491473456
2025-11-30T08:31:13.460404Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.460423Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 28 partNo 0 count 1 size 262204
2025-11-30T08:31:13.463022Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 28 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.463046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.463069Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.463079Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.463092Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 28, partNo: 0, Offset: 28 is stored on disk
2025-11-30T08:31:13.463133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.463143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.463145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.463148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.463149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.463159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.463172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.463231Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.463320Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 28
    Offset: 28
    WriteTimestampMS: 1764491473456
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.464542Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.464876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524655:3484] destroyed
2025-11-30T08:31:13.465180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524659:3486], now have 1 active actors on pipe
2025-11-30T08:31:13.465257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.465266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.465299Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|6e54cdae-471ee54-b420fdbc-949f65fe_29 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.465308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.465312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.465317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.465319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.465322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.465331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.465334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.465336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.465350Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.465364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.465491Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|6e54cdae-471ee54-b420fdbc-949f65fe_29"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|6e54cdae-471ee54-b420fdbc-949f65fe_29"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 29
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.465797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524659:3486] destroyed
2025-11-30T08:31:13.468110Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.468569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524662:3487], now have 1 active actors on pipe
2025-11-30T08:31:13.469054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.469069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.469095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 29 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.469108Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.469124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.469142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.469154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.469156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.469161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.470129Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 0
2025-11-30T08:31:13.470166Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 29 partNo 0 FormedBlobsCount 0 NewHead: Offset 29 PartNo 0 PackedSize 262214 count 1 nextOffset 30 batches 1
2025-11-30T08:31:13.470176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.470182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.470187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.470281Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 29,1 HeadOffset 29 endOffset 29 curOffset 30 d0000000000_00000000000000000029_00000_0000000001_00000? size 262204 WTime 1764491473468
2025-11-30T08:31:13.470331Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.470343Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 29 partNo 0 count 1 size 262204
2025-11-30T08:31:13.472786Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 29 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.472827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.472851Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.472862Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.472872Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 29, partNo: 0, Offset: 29 is stored on disk
2025-11-30T08:31:13.472901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.472902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.472903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.472905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.472906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.472913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.472912Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.472924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.472992Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-30T08:31:13.473477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524662:3487] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 29
    Offset: 29
    WriteTimestampMS: 1764491473468
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.474291Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.474724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524666:3489], now have 1 active actors on pipe
2025-11-30T08:31:13.474869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.474876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.474903Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|4cec2b68-b241ff4e-f22223b1-6091d7a5_30 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.474908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.474911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.474914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.474915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.474920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.474927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.474930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.474933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.474943Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.474953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.475036Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|4cec2b68-b241ff4e-f22223b1-6091d7a5_30"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.475436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524666:3489] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|4cec2b68-b241ff4e-f22223b1-6091d7a5_30"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 30
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.477745Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.478267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524669:3490], now have 1 active actors on pipe
2025-11-30T08:31:13.478689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.478702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.478727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 30 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.478739Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.478754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.478758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.478770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.478772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.478777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.479796Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 0
2025-11-30T08:31:13.479830Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 30 partNo 0 FormedBlobsCount 0 NewHead: Offset 30 PartNo 0 PackedSize 262214 count 1 nextOffset 31 batches 1
2025-11-30T08:31:13.479839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.479842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.479845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.479966Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 30,1 HeadOffset 30 endOffset 30 curOffset 31 d0000000000_00000000000000000030_00000_0000000001_00000? size 262204 WTime 1764491473479
2025-11-30T08:31:13.480014Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.480025Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 30 partNo 0 count 1 size 262204
2025-11-30T08:31:13.482657Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 30 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.482679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.482700Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.482703Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.482708Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.482716Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 30, partNo: 0, Offset: 30 is stored on disk
2025-11-30T08:31:13.482742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.482744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.482747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.482748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.482749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.482756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.482766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.482845Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.483347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524669:3490] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 30
    Offset: 30
    WriteTimestampMS: 1764491473478
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.484282Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.484809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524673:3492], now have 1 active actors on pipe
2025-11-30T08:31:13.484960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.484968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.485005Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9d124e20-793eef42-b4895906-16bcbb8a_31 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.485014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.485017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.485022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.485023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.485027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.485037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.485040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.485043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.485057Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.485070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.485135Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9d124e20-793eef42-b4895906-16bcbb8a_31"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.485494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524673:3492] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9d124e20-793eef42-b4895906-16bcbb8a_31"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 31
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.487572Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.488075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524676:3493], now have 1 active actors on pipe
2025-11-30T08:31:13.488517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.488527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.488551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 31 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.488561Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.488577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.488580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.488590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.488592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.488597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.489647Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 0
2025-11-30T08:31:13.489680Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 31 partNo 0 FormedBlobsCount 0 NewHead: Offset 31 PartNo 0 PackedSize 262214 count 1 nextOffset 32 batches 1
2025-11-30T08:31:13.489689Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.489692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.489698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.489801Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 31,1 HeadOffset 31 endOffset 31 curOffset 32 d0000000000_00000000000000000031_00000_0000000001_00000? size 262204 WTime 1764491473489
2025-11-30T08:31:13.489850Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.489863Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 31 partNo 0 count 1 size 262204
2025-11-30T08:31:13.492389Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 31
    Offset: 31
    WriteTimestampMS: 1764491473488
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.493586Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.492076Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 31 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.492099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.492118Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.492121Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.492132Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.492143Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 31, partNo: 0, Offset: 31 is stored on disk
2025-11-30T08:31:13.492182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.492185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.492187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.492189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.492190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.492200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000000_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000001_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000002_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000003_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000004_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000005_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000006_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000007_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000008_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000009_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000010_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000011_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000012_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000013_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000014_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000015_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000016_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000017_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492255Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000018_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000019_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000023_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000024_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000025_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000026_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000027_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000028_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000029_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492288Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000030_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000031_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.492294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 32 keys were taken away. Let's read 8390528 bytes
2025-11-30T08:31:13.492306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.492322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 32 blobs
2025-11-30T08:31:13.492325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000000_00000_0000000001_00000?
2025-11-30T08:31:13.492334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000001_00000_0000000001_00000?
2025-11-30T08:31:13.492346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000002_00000_0000000001_00000?
2025-11-30T08:31:13.492362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000003_00000_0000000001_00000?
2025-11-30T08:31:13.492369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000004_00000_0000000001_00000?
2025-11-30T08:31:13.492395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000005_00000_0000000001_00000?
2025-11-30T08:31:13.492403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000006_00000_0000000001_00000?
2025-11-30T08:31:13.492410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000007_00000_0000000001_00000?
2025-11-30T08:31:13.492418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000008_00000_0000000001_00000?
2025-11-30T08:31:13.492461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000009_00000_0000000001_00000?
2025-11-30T08:31:13.492468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000010_00000_0000000001_00000?
2025-11-30T08:31:13.492477Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000011_00000_0000000001_00000?
2025-11-30T08:31:13.492489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000012_00000_0000000001_00000?
2025-11-30T08:31:13.492498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000013_00000_0000000001_00000?
2025-11-30T08:31:13.492508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000014_00000_0000000001_00000?
2025-11-30T08:31:13.492517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000015_00000_0000000001_00000?
2025-11-30T08:31:13.492527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000016_00000_0000000001_00000?
2025-11-30T08:31:13.492614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000017_00000_0000000001_00000?
2025-11-30T08:31:13.492623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000018_00000_0000000001_00000?
2025-11-30T08:31:13.492632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000019_00000_0000000001_00000?
2025-11-30T08:31:13.492641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00000_0000000001_00000?
2025-11-30T08:31:13.492650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00000?
2025-11-30T08:31:13.492659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000001_00000?
2025-11-30T08:31:13.492667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000023_00000_0000000001_00000?
2025-11-30T08:31:13.492675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000024_00000_0000000001_00000?
2025-11-30T08:31:13.492684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000025_00000_0000000001_00000?
2025-11-30T08:31:13.492692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000026_00000_0000000001_00000?
2025-11-30T08:31:13.492700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000027_00000_0000000001_00000?
2025-11-30T08:31:13.492708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000028_00000_0000000001_00000?
2025-11-30T08:31:13.492716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000029_00000_0000000001_00000?
2025-11-30T08:31:13.492727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000030_00000_0000000001_00000?
2025-11-30T08:31:13.492736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000031_00000_0000000001_00000?
2025-11-30T08:31:13.492747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 32 blobs for compaction
2025-11-30T08:31:13.492756Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 1 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492760Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 1 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492763Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 2 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492766Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 3 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492769Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 4 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492772Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 5 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492776Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 6 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492779Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 7 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492783Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 8 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492786Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 9 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492805Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 10 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492810Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 11 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492813Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 12 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492815Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 13 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492818Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 14 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492821Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 15 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492825Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 16 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492827Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 17 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492831Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 18 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492834Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 19 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492837Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 20 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492840Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 21 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492843Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 22 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492845Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 23 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492849Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 24 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492852Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 25 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492855Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 26 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492858Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 27 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492861Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 28 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492864Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 29 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492867Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 30 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492870Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 31 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.492880Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 32 blobs are from cache.
2025-11-30T08:31:13.492890Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492895Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492899Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492903Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492907Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492911Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492915Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524676:3493] destroyed
2025-11-30T08:31:13.492919Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492922Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-30T08:31:13.492926Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492929Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/32] d0000000000_00000000000000000000_00000_0000000001_00000?
2025-11-30T08:31:13.492935Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.492939Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000000_00000_0000000001_00000?, parameters.CurOffset=0
2025-11-30T08:31:13.492942Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492946Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492950Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492953Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492956Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492960Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492963Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492967Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492970Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492973Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492977Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492980Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492983Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492987Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492990Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492993Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.492997Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.493004Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.493008Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.493026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 0.0/1
2025-11-30T08:31:13.493033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 0
2025-11-30T08:31:13.493056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 0 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 262214 count 1 nextOffset 1 batches 1
2025-11-30T08:31:13.493059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 0:0 appended
2025-11-30T08:31:13.493065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/32] d0000000000_00000000000000000001_00000_0000000001_00000?
2025-11-30T08:31:13.493066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.493069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000001_00000_0000000001_00000?, parameters.CurOffset=1
2025-11-30T08:31:13.493137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 1.0/1
2025-11-30T08:31:13.493140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 0
2025-11-30T08:31:13.493367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 1 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 524387 count 2 nextOffset 2 batches 1
2025-11-30T08:31:13.493371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 1:0 appended
2025-11-30T08:31:13.493376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/32] d0000000000_00000000000000000002_00000_0000000001_00000?
2025-11-30T08:31:13.493378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.493380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000002_00000_0000000001_00000?, parameters.CurOffset=2
2025-11-30T08:31:13.493440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 2.0/1
2025-11-30T08:31:13.493445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 0
2025-11-30T08:31:13.493469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 2 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 786601 count 3 nextOffset 3 batches 2
2025-11-30T08:31:13.493471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 2:0 appended
2025-11-30T08:31:13.493475Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/32] d0000000000_00000000000000000003_00000_0000000001_00000?
2025-11-30T08:31:13.493476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.493479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000003_00000_0000000001_00000?, parameters.CurOffset=3
2025-11-30T08:31:13.493546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 3.0/1
2025-11-30T08:31:13.493550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 0
2025-11-30T08:31:13.494720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 3 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1048774 count 4 nextOffset 4 batches 2
2025-11-30T08:31:13.494770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 3:0 appended
2025-11-30T08:31:13.494782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/32] d0000000000_00000000000000000004_00000_0000000001_00000?
2025-11-30T08:31:13.494784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.494787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000004_00000_0000000001_00000?, parameters.CurOffset=4
2025-11-30T08:31:13.494859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 4.0/1
2025-11-30T08:31:13.494871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 0
2025-11-30T08:31:13.494888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 4 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1310988 count 5 nextOffset 5 batches 3
2025-11-30T08:31:13.494894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 4:0 appended
2025-11-30T08:31:13.494898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/32] d0000000000_00000000000000000005_00000_0000000001_00000?
2025-11-30T08:31:13.494900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.494903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000005_00000_0000000001_00000?, parameters.CurOffset=5
2025-11-30T08:31:13.494974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 5.0/1
2025-11-30T08:31:13.494985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 0
2025-11-30T08:31:13.495581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 5 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1573161 count 6 nextOffset 6 batches 3
2025-11-30T08:31:13.495594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 5:0 appended
2025-11-30T08:31:13.495600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/32] d0000000000_00000000000000000006_00000_0000000001_00000?
2025-11-30T08:31:13.495602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.495605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000006_00000_0000000001_00000?, parameters.CurOffset=6
2025-11-30T08:31:13.495673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 6.0/1
2025-11-30T08:31:13.495683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 0
2025-11-30T08:31:13.495696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 6 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1835375 count 7 nextOffset 7 batches 4
2025-11-30T08:31:13.495702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 6:0 appended
2025-11-30T08:31:13.495706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/32] d0000000000_00000000000000000007_00000_0000000001_00000?
2025-11-30T08:31:13.495708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.495711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000007_00000_0000000001_00000?, parameters.CurOffset=7
2025-11-30T08:31:13.495781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 7.0/1
2025-11-30T08:31:13.495790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 0
2025-11-30T08:31:13.496304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 7 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2097548 count 8 nextOffset 8 batches 4
2025-11-30T08:31:13.496317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 7:0 appended
2025-11-30T08:31:13.496322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[8/32] d0000000000_00000000000000000008_00000_0000000001_00000?
2025-11-30T08:31:13.496324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.496328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000008_00000_0000000001_00000?, parameters.CurOffset=8
2025-11-30T08:31:13.496392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 8.0/1
2025-11-30T08:31:13.496402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 0
2025-11-30T08:31:13.496413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 8 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2359762 count 9 nextOffset 9 batches 5
2025-11-30T08:31:13.496420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 8:0 appended
2025-11-30T08:31:13.496423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[9/32] d0000000000_00000000000000000009_00000_0000000001_00000?
2025-11-30T08:31:13.496426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.496428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000009_00000_0000000001_00000?, parameters.CurOffset=9
2025-11-30T08:31:13.496492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 9.0/1
2025-11-30T08:31:13.496501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 0
2025-11-30T08:31:13.497010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 9 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2621935 count 10 nextOffset 10 batches 5
2025-11-30T08:31:13.497022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 9:0 appended
2025-11-30T08:31:13.497027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[10/32] d0000000000_00000000000000000010_00000_0000000001_00000?
2025-11-30T08:31:13.497032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.497034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000010_00000_0000000001_00000?, parameters.CurOffset=10
2025-11-30T08:31:13.497104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 10.0/1
2025-11-30T08:31:13.497116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 0
2025-11-30T08:31:13.497130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 10 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2884149 count 11 nextOffset 11 batches 6
2025-11-30T08:31:13.497150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 10:0 appended
2025-11-30T08:31:13.497159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[11/32] d0000000000_00000000000000000011_00000_0000000001_00000?
2025-11-30T08:31:13.497161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.497164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000011_00000_0000000001_00000?, parameters.CurOffset=11
2025-11-30T08:31:13.497223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 11.0/1
2025-11-30T08:31:13.497233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 0
2025-11-30T08:31:13.497773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 11 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3146322 count 12 nextOffset 12 batches 6
2025-11-30T08:31:13.497787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 11:0 appended
2025-11-30T08:31:13.497793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[12/32] d0000000000_00000000000000000012_00000_0000000001_00000?
2025-11-30T08:31:13.497795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.497799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000012_00000_0000000001_00000?, parameters.CurOffset=12
2025-11-30T08:31:13.497858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 12.0/1
2025-11-30T08:31:13.497869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 0
2025-11-30T08:31:13.497879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 12 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3408536 count 13 nextOffset 13 batches 7
2025-11-30T08:31:13.497881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 12:0 appended
2025-11-30T08:31:13.497885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[13/32] d0000000000_00000000000000000013_00000_0000000001_00000?
2025-11-30T08:31:13.497887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.497889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000013_00000_0000000001_00000?, parameters.CurOffset=13
2025-11-30T08:31:13.497943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 13.0/1
2025-11-30T08:31:13.497947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 0
2025-11-30T08:31:13.498480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 13 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3670709 count 14 nextOffset 14 batches 7
2025-11-30T08:31:13.498493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 13:0 appended
2025-11-30T08:31:13.498498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[14/32] d0000000000_00000000000000000014_00000_0000000001_00000?
2025-11-30T08:31:13.498500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.498503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000014_00000_0000000001_00000?, parameters.CurOffset=14
2025-11-30T08:31:13.498563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 14.0/1
2025-11-30T08:31:13.498575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 0
2025-11-30T08:31:13.498588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 14 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3932923 count 15 nextOffset 15 batches 8
2025-11-30T08:31:13.498594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 14:0 appended
2025-11-30T08:31:13.498597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[15/32] d0000000000_00000000000000000015_00000_0000000001_00000?
2025-11-30T08:31:13.498599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.498859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000015_00000_0000000001_00000?, parameters.CurOffset=15
2025-11-30T08:31:13.498940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 15.0/1
2025-11-30T08:31:13.498953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 0
2025-11-30T08:31:13.499503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 15 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4195096 count 16 nextOffset 16 batches 8
2025-11-30T08:31:13.499513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 15:0 appended
2025-11-30T08:31:13.499519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[16/32] d0000000000_00000000000000000016_00000_0000000001_00000?
2025-11-30T08:31:13.499521Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.499524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000016_00000_0000000001_00000?, parameters.CurOffset=16
2025-11-30T08:31:13.499581Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 16.0/1
2025-11-30T08:31:13.499586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 0
2025-11-30T08:31:13.499612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 16 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4457310 count 17 nextOffset 17 batches 9
2025-11-30T08:31:13.499614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 16:0 appended
2025-11-30T08:31:13.499617Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[17/32] d0000000000_00000000000000000017_00000_0000000001_00000?
2025-11-30T08:31:13.499619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.499622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000017_00000_0000000001_00000?, parameters.CurOffset=17
2025-11-30T08:31:13.499688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 17.0/1
2025-11-30T08:31:13.499691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 0
2025-11-30T08:31:13.500219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 17 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4719483 count 18 nextOffset 18 batches 9
2025-11-30T08:31:13.500237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 17:0 appended
2025-11-30T08:31:13.500243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[18/32] d0000000000_00000000000000000018_00000_0000000001_00000?
2025-11-30T08:31:13.500245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.500248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000018_00000_0000000001_00000?, parameters.CurOffset=18
2025-11-30T08:31:13.500311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 18.0/1
2025-11-30T08:31:13.500322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 0
2025-11-30T08:31:13.500336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 18 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4981697 count 19 nextOffset 19 batches 10
2025-11-30T08:31:13.500342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 18:0 appended
2025-11-30T08:31:13.500346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[19/32] d0000000000_00000000000000000019_00000_0000000001_00000?
2025-11-30T08:31:13.500348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.500353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000019_00000_0000000001_00000?, parameters.CurOffset=19
2025-11-30T08:31:13.500487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 19.0/1
2025-11-30T08:31:13.500499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 0
2025-11-30T08:31:13.501048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 19 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 5243870 count 20 nextOffset 20 batches 10
2025-11-30T08:31:13.501059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 19:0 appended
2025-11-30T08:31:13.501064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[20/32] d0000000000_00000000000000000020_00000_0000000001_00000?
2025-11-30T08:31:13.501066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.501069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000020_00000_0000000001_00000?, parameters.CurOffset=20
2025-11-30T08:31:13.501137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 20.0/1
2025-11-30T08:31:13.501141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 0
2025-11-30T08:31:13.501150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 20 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 5506084 count 21 nextOffset 21 batches 11
2025-11-30T08:31:13.501152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 20:0 appended
2025-11-30T08:31:13.501156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[21/32] d0000000000_00000000000000000021_00000_0000000001_00000?
2025-11-30T08:31:13.501157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.501160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000021_00000_0000000001_00000?, parameters.CurOffset=21
2025-11-30T08:31:13.501213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 21.0/1
2025-11-30T08:31:13.501217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 0
2025-11-30T08:31:13.501811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 21 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 5768257 count 22 nextOffset 22 batches 11
2025-11-30T08:31:13.501824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 21:0 appended
2025-11-30T08:31:13.501829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[22/32] d0000000000_00000000000000000022_00000_0000000001_00000?
2025-11-30T08:31:13.501831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.501834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000022_00000_0000000001_00000?, parameters.CurOffset=22
2025-11-30T08:31:13.501891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 22.0/1
2025-11-30T08:31:13.501896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 0
2025-11-30T08:31:13.501909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 22 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 6030471 count 23 nextOffset 23 batches 12
2025-11-30T08:31:13.501911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 22:0 appended
2025-11-30T08:31:13.501915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[23/32] d0000000000_00000000000000000023_00000_0000000001_00000?
2025-11-30T08:31:13.501917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.501919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000023_00000_0000000001_00000?, parameters.CurOffset=23
2025-11-30T08:31:13.501979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 23.0/1
2025-11-30T08:31:13.501983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 0
2025-11-30T08:31:13.502525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 23 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 6292644 count 24 nextOffset 24 batches 12
2025-11-30T08:31:13.502547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 23:0 appended
2025-11-30T08:31:13.502553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[24/32] d0000000000_00000000000000000024_00000_0000000001_00000?
2025-11-30T08:31:13.502555Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.502559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000024_00000_0000000001_00000?, parameters.CurOffset=24
2025-11-30T08:31:13.502616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 24.0/1
2025-11-30T08:31:13.502621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 0
2025-11-30T08:31:13.502633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 24 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 6554858 count 25 nextOffset 25 batches 13
2025-11-30T08:31:13.502635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 24:0 appended
2025-11-30T08:31:13.502639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[25/32] d0000000000_00000000000000000025_00000_0000000001_00000?
2025-11-30T08:31:13.502642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.502645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000025_00000_0000000001_00000?, parameters.CurOffset=25
2025-11-30T08:31:13.502702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 25.0/1
2025-11-30T08:31:13.502706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 0
2025-11-30T08:31:13.503251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 25 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 6817031 count 26 nextOffset 26 batches 13
2025-11-30T08:31:13.503255Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 25:0 appended
2025-11-30T08:31:13.503261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[26/32] d0000000000_00000000000000000026_00000_0000000001_00000?
2025-11-30T08:31:13.503263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.503265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000026_00000_0000000001_00000?, parameters.CurOffset=26
2025-11-30T08:31:13.503321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 26.0/1
2025-11-30T08:31:13.503325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 0
2025-11-30T08:31:13.503336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 26 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 7079245 count 27 nextOffset 27 batches 14
2025-11-30T08:31:13.503338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 26:0 appended
2025-11-30T08:31:13.503341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[27/32] d0000000000_00000000000000000027_00000_0000000001_00000?
2025-11-30T08:31:13.503343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.503346Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000027_00000_0000000001_00000?, parameters.CurOffset=27
2025-11-30T08:31:13.503402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 27.0/1
2025-11-30T08:31:13.503406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 0
2025-11-30T08:31:13.503980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 27 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 7341418 count 28 nextOffset 28 batches 14
2025-11-30T08:31:13.503985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 27:0 appended
2025-11-30T08:31:13.503990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[28/32] d0000000000_00000000000000000028_00000_0000000001_00000?
2025-11-30T08:31:13.503992Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.503997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000028_00000_0000000001_00000?, parameters.CurOffset=28
2025-11-30T08:31:13.504053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 28.0/1
2025-11-30T08:31:13.504057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 0
2025-11-30T08:31:13.504071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 28 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 7603632 count 29 nextOffset 29 batches 15
2025-11-30T08:31:13.504074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 28:0 appended
2025-11-30T08:31:13.504077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[29/32] d0000000000_00000000000000000029_00000_0000000001_00000?
2025-11-30T08:31:13.504079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.504082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000029_00000_0000000001_00000?, parameters.CurOffset=29
2025-11-30T08:31:13.504126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 29.0/1
2025-11-30T08:31:13.504131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 0
2025-11-30T08:31:13.504992Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 29 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 7865805 count 30 nextOffset 30 batches 15
2025-11-30T08:31:13.505003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 29:0 appended
2025-11-30T08:31:13.505008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[30/32] d0000000000_00000000000000000030_00000_0000000001_00000?
2025-11-30T08:31:13.505011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.505013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000030_00000_0000000001_00000?, parameters.CurOffset=30
2025-11-30T08:31:13.505067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 30.0/1
2025-11-30T08:31:13.505071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 0
2025-11-30T08:31:13.505147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 30 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 8128019 count 31 nextOffset 31 batches 16
2025-11-30T08:31:13.505153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 30:0 appended
2025-11-30T08:31:13.505157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[31/32] d0000000000_00000000000000000031_00000_0000000001_00000?
2025-11-30T08:31:13.505159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.505161Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000031_00000_0000000001_00000?, parameters.CurOffset=31
2025-11-30T08:31:13.505212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 31.0/1
2025-11-30T08:31:13.505216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 0
2025-11-30T08:31:13.512899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 31 partNo 0 result is x0000000000_00000000000000000000_00000_0000000031_00000 size 8128009
2025-11-30T08:31:13.512933Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1106: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000000_00000_0000000031_00000 new key d0000000000_00000000000000000000_00000_0000000031_00000 size 8128009 WTime 1764491473512
2025-11-30T08:31:13.512979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 31 partNo 0 FormedBlobsCount 1 NewHead: Offset 31 PartNo 0 PackedSize 262214 count 1 nextOffset 32 batches 1
2025-11-30T08:31:13.512988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 31:0 appended
2025-11-30T08:31:13.513121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:636: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 31,1 HeadOffset 0 endOffset 0 curOffset 32 d0000000000_00000000000000000031_00000_0000000001_00000| size 262204 WTime 1764491473512
2025-11-30T08:31:13.513211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524680:3495], now have 1 active actors on pipe
2025-11-30T08:31:13.513253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.513260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.513286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.513294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.513297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.513300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.513302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.513324Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.513336Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 31 size 8128009
2025-11-30T08:31:13.513343Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 31 partNo 0 count 1 size 262204
2025-11-30T08:31:13.513383Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|77a98fde-99b93f5e-d1cb33e0-6d048d72_32 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.513390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.513395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.513400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.513402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.513406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.513417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.513421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.513424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.513437Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.513511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.513753Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.514200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524680:3495] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|77a98fde-99b93f5e-d1cb33e0-6d048d72_32"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|77a98fde-99b93f5e-d1cb33e0-6d048d72_32"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 32
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.516884Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.517522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524686:3498], now have 1 active actors on pipe
2025-11-30T08:31:13.518255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.518274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.518300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 32 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.518315Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.518334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.518338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.518352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.518356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.518362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.519272Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 32 partNo 0
2025-11-30T08:31:13.519318Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 32 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 262214 count 1 nextOffset 33 batches 1
2025-11-30T08:31:13.519329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.519332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.519339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.519453Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 32,1 HeadOffset 32 endOffset 32 curOffset 33 d0000000000_00000000000000000032_00000_0000000001_00000? size 262204 WTime 1764491473519
2025-11-30T08:31:13.519508Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.519521Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 32 partNo 0 count 1 size 262204
2025-11-30T08:31:13.529366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 0 count 31 size 8128009 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.529389Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 31 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.529412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.529430Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 31 parts 0 suffix '0' size 8128009
2025-11-30T08:31:13.529437Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 0 suffix '124' size 262204
2025-11-30T08:31:13.529548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2279: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-30T08:31:13.529559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.530451Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 32 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.530478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.530486Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 32 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.530506Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.530524Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.530541Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 32, partNo: 0, Offset: 32 is stored on disk
2025-11-30T08:31:13.530582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.530591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.530593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.530596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.530599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.530608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000000_00000_0000000001_00000?
2025-11-30T08:31:13.530612Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000001_00000_0000000001_00000?
2025-11-30T08:31:13.530616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000002_00000_0000000001_00000?
2025-11-30T08:31:13.530620Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000003_00000_0000000001_00000?
2025-11-30T08:31:13.530623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000004_00000_0000000001_00000?
2025-11-30T08:31:13.530626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000005_00000_0000000001_00000?
2025-11-30T08:31:13.530629Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000006_00000_0000000001_00000?
2025-11-30T08:31:13.530634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000007_00000_0000000001_00000?
2025-11-30T08:31:13.530640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000008_00000_0000000001_00000?
2025-11-30T08:31:13.530644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000009_00000_0000000001_00000?
2025-11-30T08:31:13.530649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000010_00000_0000000001_00000?
2025-11-30T08:31:13.530652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000011_00000_0000000001_00000?
2025-11-30T08:31:13.530655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000012_00000_0000000001_00000?
2025-11-30T08:31:13.530658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000013_00000_0000000001_00000?
2025-11-30T08:31:13.530660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000014_00000_0000000001_00000?
2025-11-30T08:31:13.530663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000015_00000_0000000001_00000?
2025-11-30T08:31:13.530666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000016_00000_0000000001_00000?
2025-11-30T08:31:13.530668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000017_00000_0000000001_00000?
2025-11-30T08:31:13.530671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000018_00000_0000000001_00000?
2025-11-30T08:31:13.530674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000019_00000_0000000001_00000?
2025-11-30T08:31:13.530677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000020_00000_0000000001_00000?
2025-11-30T08:31:13.530680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000021_00000_0000000001_00000?
2025-11-30T08:31:13.530684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000022_00000_0000000001_00000?
2025-11-30T08:31:13.530686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000023_00000_0000000001_00000?
2025-11-30T08:31:13.530689Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000024_00000_0000000001_00000?
2025-11-30T08:31:13.530692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000025_00000_0000000001_00000?
2025-11-30T08:31:13.530694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000026_00000_0000000001_00000?
2025-11-30T08:31:13.530697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000027_00000_0000000001_00000?
2025-11-30T08:31:13.530699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000028_00000_0000000001_00000?
2025-11-30T08:31:13.530702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000029_00000_0000000001_00000?
2025-11-30T08:31:13.530704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000030_00000_0000000001_00000?
2025-11-30T08:31:13.530707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000031_00000_0000000001_00000?
2025-11-30T08:31:13.530733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.530750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.530770Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.530784Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000000_00000_0000000001_00000?(+) to d0000000000_00000000000000000000_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530793Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000001_00000_0000000001_00000?(+) to d0000000000_00000000000000000001_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530795Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000002_00000_0000000001_00000?(+) to d0000000000_00000000000000000002_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530797Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000003_00000_0000000001_00000?(+) to d0000000000_00000000000000000003_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530800Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000004_00000_0000000001_00000?(+) to d0000000000_00000000000000000004_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530803Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000005_00000_0000000001_00000?(+) to d0000000000_00000000000000000005_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530805Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000006_00000_0000000001_00000?(+) to d0000000000_00000000000000000006_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530806Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000007_00000_0000000001_00000?(+) to d0000000000_00000000000000000007_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530808Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000008_00000_0000000001_00000?(+) to d0000000000_00000000000000000008_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530811Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000009_00000_0000000001_00000?(+) to d0000000000_00000000000000000009_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530813Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000010_00000_0000000001_00000?(+) to d0000000000_00000000000000000010_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530815Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000011_00000_0000000001_00000?(+) to d0000000000_00000000000000000011_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530817Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000012_00000_0000000001_00000?(+) to d0000000000_00000000000000000012_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530819Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000013_00000_0000000001_00000?(+) to d0000000000_00000000000000000013_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530821Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000014_00000_0000000001_00000?(+) to d0000000000_00000000000000000014_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530824Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000015_00000_0000000001_00000?(+) to d0000000000_00000000000000000015_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530826Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000016_00000_0000000001_00000?(+) to d0000000000_00000000000000000016_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530827Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000017_00000_0000000001_00000?(+) to d0000000000_00000000000000000017_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530829Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000018_00000_0000000001_00000?(+) to d0000000000_00000000000000000018_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530831Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000019_00000_0000000001_00000?(+) to d0000000000_00000000000000000019_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530833Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000020_00000_0000000001_00000?(+) to d0000000000_00000000000000000020_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530835Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000021_00000_0000000001_00000?(+) to d0000000000_00000000000000000021_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530837Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000022_00000_0000000001_00000?(+) to d0000000000_00000000000000000022_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530839Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000023_00000_0000000001_00000?(+) to d0000000000_00000000000000000023_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530841Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000024_00000_0000000001_00000?(+) to d0000000000_00000000000000000024_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530843Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000025_00000_0000000001_00000?(+) to d0000000000_00000000000000000025_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530845Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000026_00000_0000000001_00000?(+) to d0000000000_00000000000000000026_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530846Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000027_00000_0000000001_00000?(+) to d0000000000_00000000000000000027_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530848Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000028_00000_0000000001_00000?(+) to d0000000000_00000000000000000028_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530850Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000029_00000_0000000001_00000?(+) to d0000000000_00000000000000000029_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530852Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000030_00000_0000000001_00000?(+) to d0000000000_00000000000000000030_00000_0000000001_00000?(+)
2025-11-30T08:31:13.530854Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000031_00000_0000000001_00000?(+) to d0000000000_00000000000000000031_00000_0000000001_00000?(+)
2025-11-30T08:31:13.531212Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.531803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524686:3498] destroyed
2025-11-30T08:31:13.532068Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 0 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532080Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 1 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532085Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 2 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532089Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 3 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532094Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 4 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532098Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 5 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532103Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 6 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532107Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 7 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532112Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 8 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532116Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 9 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532120Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 10 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532124Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 11 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532128Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 12 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532132Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 13 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532136Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 14 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532140Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 15 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532146Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 16 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532151Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 17 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532155Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 18 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532159Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 19 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532163Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 20 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532167Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 21 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532171Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 22 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532175Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 23 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532179Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 24 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532183Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 25 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532187Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 26 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532191Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 27 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532195Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 28 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532198Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 29 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532203Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 30 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.532206Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 31 count 1 actorID [2:7578433167081267540:2364]
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 32
    Offset: 32
    WriteTimestampMS: 1764491473518
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 11
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:308592025-11-30T08:31:13.532222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.532240Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.532247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.532249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.532251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.532253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.532256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.532263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.532345Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532350Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532354Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532358Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532362Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532367Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532371Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532375Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532379Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532383Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532387Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532390Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532393Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532397Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532400Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532403Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532407Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532410Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532414Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532417Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532421Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532424Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532427Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532431Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532434Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532437Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532440Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532444Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532448Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532451Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532455Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.532458Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.533277Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.534111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524692:3502], now have 1 active actors on pipe
2025-11-30T08:31:13.534155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.534161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.534196Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f88bb9c0-5710be29-152ce29d-a5895725_33 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.534209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.534213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.534218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.534221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.534224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.534234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.534242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.534244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.534260Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.534278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.534346Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.534650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524692:3502] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f88bb9c0-5710be29-152ce29d-a5895725_33"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f88bb9c0-5710be29-152ce29d-a5895725_33"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 33
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.537368Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.537886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524696:3504], now have 1 active actors on pipe
2025-11-30T08:31:13.538292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.538309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.538335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 33 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.538353Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.538373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.538382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.538394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.538400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.538405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.539347Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 33 partNo 0
2025-11-30T08:31:13.539393Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 33 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 262214 count 1 nextOffset 34 batches 1
2025-11-30T08:31:13.539403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.539413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.539420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.539528Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 33,1 HeadOffset 33 endOffset 33 curOffset 34 d0000000000_00000000000000000033_00000_0000000001_00000? size 262204 WTime 1764491473539
2025-11-30T08:31:13.539577Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.539590Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 33 partNo 0 count 1 size 262204
2025-11-30T08:31:13.545004Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 33 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.545039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.545069Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.545070Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 33 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.545089Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.545105Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 33, partNo: 0, Offset: 33 is stored on disk
2025-11-30T08:31:13.545152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.545159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.545162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.545165Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.545167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.545178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.545197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.545325Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-30T08:31:13.545780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524696:3504] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 33
    Offset: 33
    WriteTimestampMS: 1764491473538
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.546482Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.546884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524714:3514], now have 1 active actors on pipe
2025-11-30T08:31:13.547039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.547051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.547092Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9fe04487-b691c998-200b5a9e-bd503e03_34 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.547106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.547115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.547121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.547124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.547129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.547140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.547143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.547146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.547164Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.547180Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.547259Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9fe04487-b691c998-200b5a9e-bd503e03_34"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.547621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524714:3514] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9fe04487-b691c998-200b5a9e-bd503e03_34"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 34
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.549745Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.550187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524717:3515], now have 1 active actors on pipe
2025-11-30T08:31:13.550634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.550650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.550676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 34 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.550692Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.550708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.550716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.550727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.550730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.550737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.551666Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 34 partNo 0
2025-11-30T08:31:13.551718Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 34 partNo 0 FormedBlobsCount 0 NewHead: Offset 34 PartNo 0 PackedSize 262214 count 1 nextOffset 35 batches 1
2025-11-30T08:31:13.551732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.551736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.551738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.551866Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 34,1 HeadOffset 34 endOffset 34 curOffset 35 d0000000000_00000000000000000034_00000_0000000001_00000? size 262204 WTime 1764491473551
2025-11-30T08:31:13.552132Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.552155Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 34 partNo 0 count 1 size 262204
2025-11-30T08:31:13.554319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 34 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.554346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.554363Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 34 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.554370Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.554380Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.554393Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 34, partNo: 0, Offset: 34 is stored on disk
2025-11-30T08:31:13.554435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.554442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.554445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.554447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.554449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.554463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.554477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.554548Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-30T08:31:13.555013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524717:3515] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 34
    Offset: 34
    WriteTimestampMS: 1764491473549
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 2
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.556014Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.556562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524721:3517], now have 1 active actors on pipe
2025-11-30T08:31:13.556583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.556590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.556623Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|65e3b411-92ba4800-6d4dad82-8df6aff9_35 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.556637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.556641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.556646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.556648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.556652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.556663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.556670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.556673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.556689Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.556709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.556830Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|65e3b411-92ba4800-6d4dad82-8df6aff9_35"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.557229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524721:3517] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|65e3b411-92ba4800-6d4dad82-8df6aff9_35"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 35
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.559354Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.560012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524724:3518], now have 1 active actors on pipe
2025-11-30T08:31:13.560158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.560170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.560194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 35 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.560209Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.560226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.560234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.560244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.560246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.560251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.561297Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 35 partNo 0
2025-11-30T08:31:13.561341Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 35 partNo 0 FormedBlobsCount 0 NewHead: Offset 35 PartNo 0 PackedSize 262214 count 1 nextOffset 36 batches 1
2025-11-30T08:31:13.561356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.561359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.561362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.561473Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 35,1 HeadOffset 35 endOffset 35 curOffset 36 d0000000000_00000000000000000035_00000_0000000001_00000? size 262204 WTime 1764491473561
2025-11-30T08:31:13.561541Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.561562Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 35 partNo 0 count 1 size 262204
2025-11-30T08:31:13.566885Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 35
    Offset: 35
    WriteTimestampMS: 1764491473559
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.566566Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 35 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.566589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.566612Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.566622Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.566632Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 35, partNo: 0, Offset: 35 is stored on disk
2025-11-30T08:31:13.566667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.566669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.566671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.566673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.566675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.566683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.566694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.566797Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 35 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.567727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524724:3518] destroyed
2025-11-30T08:31:13.568200Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.568807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524728:3520], now have 1 active actors on pipe
2025-11-30T08:31:13.568828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.568835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.568870Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|e9ea77c5-afa6c979-4955ef1b-2f0591b0_36 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.568881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.568885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.568890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.568892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.568896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.568906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.568909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.568912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.568931Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.569050Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.568949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|e9ea77c5-afa6c979-4955ef1b-2f0591b0_36"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.569872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524728:3520] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|e9ea77c5-afa6c979-4955ef1b-2f0591b0_36"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 36
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.571799Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.574008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524731:3521], now have 1 active actors on pipe
2025-11-30T08:31:13.574427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.574442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.574468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 36 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.574483Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.574501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.574507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.574517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.574519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.574524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.575473Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 36 partNo 0
2025-11-30T08:31:13.575512Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 36 partNo 0 FormedBlobsCount 0 NewHead: Offset 36 PartNo 0 PackedSize 262214 count 1 nextOffset 37 batches 1
2025-11-30T08:31:13.575524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.575527Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.575530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.575631Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 36,1 HeadOffset 36 endOffset 36 curOffset 37 d0000000000_00000000000000000036_00000_0000000001_00000? size 262204 WTime 1764491473575
2025-11-30T08:31:13.575680Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.575692Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 36 partNo 0 count 1 size 262204
2025-11-30T08:31:13.585541Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 36 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.585572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.585600Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.585612Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.585627Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 36, partNo: 0, Offset: 36 is stored on disk
2025-11-30T08:31:13.585682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.585685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.585687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.585691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.585694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.585705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.585719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.585787Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 36 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.586309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524731:3521] destroyed
2025-11-30T08:31:13.585883Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 36
    Offset: 36
    WriteTimestampMS: 1764491473574
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 10
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.591923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524739:3523], now have 1 active actors on pipe
2025-11-30T08:31:13.590269Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.592143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.592153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.592195Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|4e6b7863-dc3f2e44-5f0a59c9-62f9458f_37 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.592210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.592216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.592223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.592225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.592229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.592240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.592244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.592247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.592266Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.592286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.592476Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.592971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524739:3523] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|4e6b7863-dc3f2e44-5f0a59c9-62f9458f_37"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|4e6b7863-dc3f2e44-5f0a59c9-62f9458f_37"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 37
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.597971Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.598599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524758:3531], now have 1 active actors on pipe
2025-11-30T08:31:13.599213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.599225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.599244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 37 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.599256Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.599269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.599272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.599283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.599285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.599289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.600397Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 37 partNo 0
2025-11-30T08:31:13.600433Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 37 partNo 0 FormedBlobsCount 0 NewHead: Offset 37 PartNo 0 PackedSize 262214 count 1 nextOffset 38 batches 1
2025-11-30T08:31:13.600442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.600444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.600448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.600546Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 37,1 HeadOffset 37 endOffset 37 curOffset 38 d0000000000_00000000000000000037_00000_0000000001_00000? size 262204 WTime 1764491473600
2025-11-30T08:31:13.600589Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.600601Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 37 partNo 0 count 1 size 262204
2025-11-30T08:31:13.609837Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 37 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.609858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.609881Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.609891Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.609904Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 37, partNo: 0, Offset: 37 is stored on disk
2025-11-30T08:31:13.609949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.609952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.609956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.609960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.609961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.609968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.609978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.610105Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 37 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.610193Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-30T08:31:13.610743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524758:3531] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 37
    Offset: 37
    WriteTimestampMS: 1764491473599
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 9
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.611549Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.613050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524762:3533], now have 1 active actors on pipe
2025-11-30T08:31:13.613198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.613213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.613246Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ba2218e2-89c60c1d-66331591-86588092_38 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.613256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.613259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.613264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.613265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.613269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.613278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.613281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.613283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.613297Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.613314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.613475Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ba2218e2-89c60c1d-66331591-86588092_38"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.613870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524762:3533] destroyed
2025-11-30T08:31:13.613885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.613887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.613889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.613892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.613894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ba2218e2-89c60c1d-66331591-86588092_38"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 38
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.618798Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.621343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524765:3534], now have 1 active actors on pipe
2025-11-30T08:31:13.625761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.625791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.625819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 38 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.625835Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.625854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.625867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.625879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.625882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.625886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.626828Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 38 partNo 0
2025-11-30T08:31:13.626884Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 38 partNo 0 FormedBlobsCount 0 NewHead: Offset 38 PartNo 0 PackedSize 262214 count 1 nextOffset 39 batches 1
2025-11-30T08:31:13.626896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.626907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.626911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.627028Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 38,1 HeadOffset 38 endOffset 38 curOffset 39 d0000000000_00000000000000000038_00000_0000000001_00000? size 262204 WTime 1764491473625
2025-11-30T08:31:13.627096Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.627121Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 38 partNo 0 count 1 size 262204
2025-11-30T08:31:13.645216Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 38 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.645252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.645283Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.645297Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.645313Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 38, partNo: 0, Offset: 38 is stored on disk
2025-11-30T08:31:13.645572Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.645371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.645379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.645382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.645385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.645388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.645402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.645420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.645483Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 38 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.646154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524765:3534] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 38
    Offset: 38
    WriteTimestampMS: 1764491473625
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 19
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.647990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524777:3538], now have 1 active actors on pipe
2025-11-30T08:31:13.648225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.648238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.648278Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|21a6ef9f-97a19952-d42bad95-affb556_39 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.648292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.648296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.648304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.648306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.648311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.648323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.648327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.648330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.648350Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.648369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
2025-11-30T08:31:13.648961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524777:3538] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|21a6ef9f-97a19952-d42bad95-affb556_39"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.647386Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.648534Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|21a6ef9f-97a19952-d42bad95-affb556_39"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 39
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.652109Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.652600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524780:3539], now have 1 active actors on pipe
2025-11-30T08:31:13.653100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.653115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.653202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 39 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.653216Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.653234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.653239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.653252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.653254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.653259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.654410Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 39 partNo 0
2025-11-30T08:31:13.654457Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 39 partNo 0 FormedBlobsCount 0 NewHead: Offset 39 PartNo 0 PackedSize 262214 count 1 nextOffset 40 batches 1
2025-11-30T08:31:13.654468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.654472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.654476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.654585Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 39,1 HeadOffset 39 endOffset 39 curOffset 40 d0000000000_00000000000000000039_00000_0000000001_00000? size 262204 WTime 1764491473654
2025-11-30T08:31:13.654641Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.654653Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 39 partNo 0 count 1 size 262204
2025-11-30T08:31:13.666107Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 39 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.666149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.666176Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.666181Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 39 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.666188Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.666200Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 39, partNo: 0, Offset: 39 is stored on disk
2025-11-30T08:31:13.666245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.666256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.666258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.666261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.666263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.666272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.666286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.666392Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.667379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524780:3539] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 39
    Offset: 39
    WriteTimestampMS: 1764491473653
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 11
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.669679Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.670305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524784:3541], now have 1 active actors on pipe
2025-11-30T08:31:13.670810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.670826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.670871Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ff06e853-cb55886-87a16176-d190b870_40 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.670885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.670889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.670894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.670897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.670901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.670910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.670914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.670917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.670930Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.670944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.671040Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.671340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524784:3541] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ff06e853-cb55886-87a16176-d190b870_40"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ff06e853-cb55886-87a16176-d190b870_40"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 40
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.674438Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.675004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524788:3543], now have 1 active actors on pipe
2025-11-30T08:31:13.675376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.675392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.675416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 40 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.675427Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.675440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.675443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.675452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.675455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.675459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.676647Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 40 partNo 0
2025-11-30T08:31:13.676688Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 40 partNo 0 FormedBlobsCount 0 NewHead: Offset 40 PartNo 0 PackedSize 262214 count 1 nextOffset 41 batches 1
2025-11-30T08:31:13.676696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.676699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.676703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.676817Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 40,1 HeadOffset 40 endOffset 40 curOffset 41 d0000000000_00000000000000000040_00000_0000000001_00000? size 262204 WTime 1764491473676
2025-11-30T08:31:13.676873Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.676891Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 40 partNo 0 count 1 size 262204
2025-11-30T08:31:13.681733Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 40 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.681756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.681775Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.681784Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.681796Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 40, partNo: 0, Offset: 40 is stored on disk
2025-11-30T08:31:13.681829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.681832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.681834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.681837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.681839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.681848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.681859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.681911Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 40 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.681990Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-30T08:31:13.682378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524788:3543] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 40
    Offset: 40
    WriteTimestampMS: 1764491473674
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.682983Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.683344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524792:3545], now have 1 active actors on pipe
2025-11-30T08:31:13.683423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.683429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.683458Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ab7bef1d-8f809ed8-c1eb03a6-4947014b_41 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.683464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.683467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.683471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.683473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.683476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.683486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.683489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.683491Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.683502Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.683511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.683891Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.684122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524792:3545] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ab7bef1d-8f809ed8-c1eb03a6-4947014b_41"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ab7bef1d-8f809ed8-c1eb03a6-4947014b_41"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 41
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.686472Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.686887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524795:3546], now have 1 active actors on pipe
2025-11-30T08:31:13.687370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.687386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.687406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 41 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.687421Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.687436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.687443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.687452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.687458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.687463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.688525Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 41 partNo 0
2025-11-30T08:31:13.688568Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 41 partNo 0 FormedBlobsCount 0 NewHead: Offset 41 PartNo 0 PackedSize 262214 count 1 nextOffset 42 batches 1
2025-11-30T08:31:13.688582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.688585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.688588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.688683Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 41,1 HeadOffset 41 endOffset 41 curOffset 42 d0000000000_00000000000000000041_00000_0000000001_00000? size 262204 WTime 1764491473688
2025-11-30T08:31:13.688730Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.688746Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 41 partNo 0 count 1 size 262204
2025-11-30T08:31:13.691688Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 41 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.691707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.691723Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.691730Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.691736Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 41 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.691738Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 41, partNo: 0, Offset: 41 is stored on disk
2025-11-30T08:31:13.691766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.691768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.691770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.691773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.691775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.691781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.691791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.691915Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 41
    Offset: 41
    WriteTimestampMS: 1764491473687
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.692321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524795:3546] destroyed
2025-11-30T08:31:13.692833Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.693204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524802:3548], now have 1 active actors on pipe
2025-11-30T08:31:13.693342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.693354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.693378Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|c8bf7bfc-20411354-b47b5f0d-5cf87b30_42 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.693388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.693392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.693397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.693403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.693406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.693414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.693417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.693420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.693435Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.693455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.693541Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|c8bf7bfc-20411354-b47b5f0d-5cf87b30_42"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.694114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524802:3548] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|c8bf7bfc-20411354-b47b5f0d-5cf87b30_42"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 42
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.695808Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.696412Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524805:3549], now have 1 active actors on pipe
2025-11-30T08:31:13.696907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.696920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.696939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 42 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.696949Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.696959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.696963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.696971Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.696974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.696978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.697915Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 42 partNo 0
2025-11-30T08:31:13.697945Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 42 partNo 0 FormedBlobsCount 0 NewHead: Offset 42 PartNo 0 PackedSize 262214 count 1 nextOffset 43 batches 1
2025-11-30T08:31:13.697953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.697956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.697959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.698055Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 42,1 HeadOffset 42 endOffset 42 curOffset 43 d0000000000_00000000000000000042_00000_0000000001_00000? size 262204 WTime 1764491473698
2025-11-30T08:31:13.698099Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.698112Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 42 partNo 0 count 1 size 262204
2025-11-30T08:31:13.700150Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 42 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.700168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.700183Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.700192Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.700193Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 42 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.700205Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 42, partNo: 0, Offset: 42 is stored on disk
2025-11-30T08:31:13.700230Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.700235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.700237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.700239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.700240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.700245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.700253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.700333Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.700621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524805:3549] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 42
    Offset: 42
    WriteTimestampMS: 1764491473696
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.701173Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.701524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524809:3551], now have 1 active actors on pipe
2025-11-30T08:31:13.701624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.701632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.701656Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|93d257ff-986de158-ed4ade0-d094f164_43 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.701665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.701668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.701672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.701674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.701678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.701684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.701686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.701689Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.701697Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.701704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.701784Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|93d257ff-986de158-ed4ade0-d094f164_43"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|93d257ff-986de158-ed4ade0-d094f164_43"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 43
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.703025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524809:3551] destroyed
2025-11-30T08:31:13.703997Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.704333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524812:3552], now have 1 active actors on pipe
2025-11-30T08:31:13.704680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.704688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.704704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 43 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.704715Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.704723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.704729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.704737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.704742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.704745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.705913Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 43 partNo 0
2025-11-30T08:31:13.705945Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 43 partNo 0 FormedBlobsCount 0 NewHead: Offset 43 PartNo 0 PackedSize 262214 count 1 nextOffset 44 batches 1
2025-11-30T08:31:13.705951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.705954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.705957Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.706037Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 43,1 HeadOffset 43 endOffset 43 curOffset 44 d0000000000_00000000000000000043_00000_0000000001_00000? size 262204 WTime 1764491473705
2025-11-30T08:31:13.706080Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.706096Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 43 partNo 0 count 1 size 262204
2025-11-30T08:31:13.708997Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 43 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.709017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.709033Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.709039Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.709048Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 43, partNo: 0, Offset: 43 is stored on disk
2025-11-30T08:31:13.709060Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 43 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.709074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.709076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.709078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.709081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.709083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.709088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.709098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.709208Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 43
    Offset: 43
    WriteTimestampMS: 1764491473704
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.710002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524812:3552] destroyed
2025-11-30T08:31:13.710166Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.710827Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.710524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524817:3555], now have 1 active actors on pipe
CallPersQueueGRPC response:
2025-11-30T08:31:13.710667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.710671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.710691Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|8eb3116f-84d47bcb-c03637e9-debc6f0a_44 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.710697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.710700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.710703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|8eb3116f-84d47bcb-c03637e9-debc6f0a_44"
    Status: Active
    SeqNo: 0
  }
}
2025-11-30T08:31:13.710705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.710708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.710714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.710716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.710718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.710727Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.710735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.711536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524817:3555] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|8eb3116f-84d47bcb-c03637e9-debc6f0a_44"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 44
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.713224Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.713600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524820:3556], now have 1 active actors on pipe
2025-11-30T08:31:13.713994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.714006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.714025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 44 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.714037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.714040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.714043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.714046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.714048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.714055Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.714064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.714075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.714081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.714083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.714087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.715188Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 44 partNo 0
2025-11-30T08:31:13.715226Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 44 partNo 0 FormedBlobsCount 0 NewHead: Offset 44 PartNo 0 PackedSize 262214 count 1 nextOffset 45 batches 1
2025-11-30T08:31:13.715240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.715242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.715246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.715342Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 44,1 HeadOffset 44 endOffset 44 curOffset 45 d0000000000_00000000000000000044_00000_0000000001_00000? size 262204 WTime 1764491473714
2025-11-30T08:31:13.715385Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.715398Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 44 partNo 0 count 1 size 262204
2025-11-30T08:31:13.721801Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 44 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.721829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.721847Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 44 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.721849Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.721860Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.721870Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 44, partNo: 0, Offset: 44 is stored on disk
2025-11-30T08:31:13.721908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.721916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.721919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.721921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.721923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.721930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.721943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.722010Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 44
    Offset: 44
    WriteTimestampMS: 1764491473713
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 6
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.722788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524820:3556] destroyed
2025-11-30T08:31:13.723321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524824:3558], now have 1 active actors on pipe
2025-11-30T08:31:13.723480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.723487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.723507Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|6219fd90-f8b43ff9-5ece089e-3db097e_45 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.723513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.723516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.723519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.723521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.723523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.723529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.723531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.723533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.723543Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.723551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|6219fd90-f8b43ff9-5ece089e-3db097e_45"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|6219fd90-f8b43ff9-5ece089e-3db097e_45"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 45
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.724818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524824:3558] destroyed
2025-11-30T08:31:13.726438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524827:3559], now have 1 active actors on pipe
2025-11-30T08:31:13.726823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.726832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.726850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 45 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.726858Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.726867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.726873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.726880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.726882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.726885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.728069Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 45 partNo 0
2025-11-30T08:31:13.728099Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 45 partNo 0 FormedBlobsCount 0 NewHead: Offset 45 PartNo 0 PackedSize 262214 count 1 nextOffset 46 batches 1
2025-11-30T08:31:13.728105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.728107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.728110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.728191Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 45,1 HeadOffset 45 endOffset 45 curOffset 46 d0000000000_00000000000000000045_00000_0000000001_00000? size 262204 WTime 1764491473726
2025-11-30T08:31:13.728229Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.728239Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 45 partNo 0 count 1 size 262204
2025-11-30T08:31:13.722916Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.723652Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.726008Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.730210Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 45 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.730223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.730239Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.730246Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.730254Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 45, partNo: 0, Offset: 45 is stored on disk
2025-11-30T08:31:13.730277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.730280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.730281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.730283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.730285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.730290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.730299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.730389Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 45 partno 0 count 1 parts 0 suffix '63' size 262204
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 45
    Offset: 45
    WriteTimestampMS: 1764491473726
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.731827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524827:3559] destroyed
2025-11-30T08:31:13.731837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524831:3561], now have 1 active actors on pipe
2025-11-30T08:31:13.731844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.731859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.731880Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f20e04dd-8dc3364-41cde3fd-a39faf73_46 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.731885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.731888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.731892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.731894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.731896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.731902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.731905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.731907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.731915Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.731924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f20e04dd-8dc3364-41cde3fd-a39faf73_46"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.730469Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.731279Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.732048Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f20e04dd-8dc3364-41cde3fd-a39faf73_46"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 46
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.734943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524831:3561] destroyed
2025-11-30T08:31:13.734958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524834:3562], now have 1 active actors on pipe
2025-11-30T08:31:13.735152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.735162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.735178Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 46 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.735185Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.735193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.735195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.735203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.735204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.735207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.734387Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.736371Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 46 partNo 0
2025-11-30T08:31:13.736404Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 46 partNo 0 FormedBlobsCount 0 NewHead: Offset 46 PartNo 0 PackedSize 262214 count 1 nextOffset 47 batches 1
2025-11-30T08:31:13.736410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.736414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.736417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.736497Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 46,1 HeadOffset 46 endOffset 46 curOffset 47 d0000000000_00000000000000000046_00000_0000000001_00000? size 262204 WTime 1764491473735
2025-11-30T08:31:13.736534Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.736546Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 46 partNo 0 count 1 size 262204
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 46
    Offset: 46
    WriteTimestampMS: 1764491473734
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.741194Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 46 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.741215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.741233Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.741242Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.741251Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 46, partNo: 0, Offset: 46 is stored on disk
2025-11-30T08:31:13.741280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.741282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.741284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.741287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.741289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.741295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.741304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.741357Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 46 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.741437Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.742489Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.743102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524834:3562] destroyed
2025-11-30T08:31:13.743124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524842:3565], now have 1 active actors on pipe
2025-11-30T08:31:13.743131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.743136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.743156Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|bc1267e0-7bbd5c60-15c8aa12-57418fe0_47 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.743162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.743165Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.743168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.743170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.743173Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.743179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.743182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.743184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.743193Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.743202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.743274Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|bc1267e0-7bbd5c60-15c8aa12-57418fe0_47"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|bc1267e0-7bbd5c60-15c8aa12-57418fe0_47"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 47
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.745873Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.746871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524842:3565] destroyed
2025-11-30T08:31:13.746884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524845:3567], now have 1 active actors on pipe
2025-11-30T08:31:13.746965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.746970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.746986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 47 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.746995Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.747004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.747008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.747014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.747016Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.747019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.748178Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 47 partNo 0
2025-11-30T08:31:13.748207Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 47 partNo 0 FormedBlobsCount 0 NewHead: Offset 47 PartNo 0 PackedSize 262214 count 1 nextOffset 48 batches 1
2025-11-30T08:31:13.748213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.748215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.748218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.748299Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 47,1 HeadOffset 47 endOffset 47 curOffset 48 d0000000000_00000000000000000047_00000_0000000001_00000? size 262204 WTime 1764491473746
2025-11-30T08:31:13.748335Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.748347Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 47 partNo 0 count 1 size 262204
2025-11-30T08:31:13.750731Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 47 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.750756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.750776Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.750789Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.750796Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 47 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.750799Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 47, partNo: 0, Offset: 47 is stored on disk
2025-11-30T08:31:13.750832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.750835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.750837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.750840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.750841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.750849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.750860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.750927Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.751362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524845:3567] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 47
    Offset: 47
    WriteTimestampMS: 1764491473746
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.751876Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.752173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524849:3569], now have 1 active actors on pipe
2025-11-30T08:31:13.752347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.752354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.752376Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9582d21a-d29fcc64-edef95df-50f25dd0_48 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.752382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.752385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.752389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.752390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.752394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.752400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.752403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.752405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.752414Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.752421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.752569Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.752849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524849:3569] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9582d21a-d29fcc64-edef95df-50f25dd0_48"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9582d21a-d29fcc64-edef95df-50f25dd0_48"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 48
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.754888Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.755216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524852:3570], now have 1 active actors on pipe
2025-11-30T08:31:13.755659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.755672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.755702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 48 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.755710Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.755720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.755723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.755732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.755734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.755738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.756752Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 48 partNo 0
2025-11-30T08:31:13.756803Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 48 partNo 0 FormedBlobsCount 0 NewHead: Offset 48 PartNo 0 PackedSize 262214 count 1 nextOffset 49 batches 1
2025-11-30T08:31:13.756812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.756814Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.756818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.756913Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 48,1 HeadOffset 48 endOffset 48 curOffset 49 d0000000000_00000000000000000048_00000_0000000001_00000? size 262204 WTime 1764491473756
2025-11-30T08:31:13.756959Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.756971Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 48 partNo 0 count 1 size 262204
2025-11-30T08:31:13.761744Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 48 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.761770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.761789Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.761796Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 48 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.761801Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.761810Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 48, partNo: 0, Offset: 48 is stored on disk
2025-11-30T08:31:13.761846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.761854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.761857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.761859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.761861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.761868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.761881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.761948Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.762413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524852:3570] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 48
    Offset: 48
    WriteTimestampMS: 1764491473754
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.762970Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.763462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524857:3573], now have 1 active actors on pipe
2025-11-30T08:31:13.763590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.763597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.763619Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|8fcc1753-e5ac039b-a4378307-fc152fba_49 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.763625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.763627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.763630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.763632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.763634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.763640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.763643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.763645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.763654Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.763663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.763794Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.764114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524857:3573] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|8fcc1753-e5ac039b-a4378307-fc152fba_49"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|8fcc1753-e5ac039b-a4378307-fc152fba_49"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 49
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.766216Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.766659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524860:3574], now have 1 active actors on pipe
2025-11-30T08:31:13.767075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.767083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.767098Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 49 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.767105Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.767114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.767121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.767128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.767130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.767133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.768313Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 49 partNo 0
2025-11-30T08:31:13.768338Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 49 partNo 0 FormedBlobsCount 0 NewHead: Offset 49 PartNo 0 PackedSize 262214 count 1 nextOffset 50 batches 1
2025-11-30T08:31:13.768344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.768347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.768350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.768431Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 49,1 HeadOffset 49 endOffset 49 curOffset 50 d0000000000_00000000000000000049_00000_0000000001_00000? size 262204 WTime 1764491473768
2025-11-30T08:31:13.768467Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.768478Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 49 partNo 0 count 1 size 262204
2025-11-30T08:31:13.774032Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 49 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.774051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.774068Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.774077Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.774086Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 49, partNo: 0, Offset: 49 is stored on disk
2025-11-30T08:31:13.774113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.774116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.774118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.774120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.774122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.774129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.774139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.774190Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 49 partno 0 count 1 parts 0 suffix '63' size 262204
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 49
    Offset: 49
    WriteTimestampMS: 1764491473767
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.775881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524860:3574] destroyed
2025-11-30T08:31:13.775893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524864:3576], now have 1 active actors on pipe
2025-11-30T08:31:13.775901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.775907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.775928Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|3d1d3585-86b223bc-850abb08-4fdcc031_50 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.775934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.775937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.775941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.775944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.775956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.775963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.775965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.775968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.775977Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.775986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|3d1d3585-86b223bc-850abb08-4fdcc031_50"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.774260Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.775304Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.776129Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|3d1d3585-86b223bc-850abb08-4fdcc031_50"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 50
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.777392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524864:3576] destroyed
2025-11-30T08:31:13.778443Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.778965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524867:3577], now have 1 active actors on pipe
2025-11-30T08:31:13.779096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.779102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.779118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 50 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.779124Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.779133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.779136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.779142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.779144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.779148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.780307Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 50 partNo 0
2025-11-30T08:31:13.780335Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 50 partNo 0 FormedBlobsCount 0 NewHead: Offset 50 PartNo 0 PackedSize 262214 count 1 nextOffset 51 batches 1
2025-11-30T08:31:13.780340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.780343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.780345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.780426Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 50,1 HeadOffset 50 endOffset 50 curOffset 51 d0000000000_00000000000000000050_00000_0000000001_00000? size 262204 WTime 1764491473777
2025-11-30T08:31:13.780460Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.780471Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 50 partNo 0 count 1 size 262204
2025-11-30T08:31:13.782241Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 50 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.782260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.782276Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 50 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.782276Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.782284Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.782291Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 50, partNo: 0, Offset: 50 is stored on disk
2025-11-30T08:31:13.782314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.782316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.782318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.782321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.782323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.782329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.782337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.782490Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.782706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524867:3577] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 50
    Offset: 50
    WriteTimestampMS: 1764491473777
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.783740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524871:3579], now have 1 active actors on pipe
2025-11-30T08:31:13.783751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.783755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.783777Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|910f25f9-ffc7e825-d2c9bf46-d9816ca6_51 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.783782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.783785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.783788Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.783790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.783792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.783797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.783799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.783801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.783809Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.783817Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.784083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524871:3579] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|910f25f9-ffc7e825-d2c9bf46-d9816ca6_51"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|910f25f9-ffc7e825-d2c9bf46-d9816ca6_51"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 51
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.786538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524874:3580], now have 1 active actors on pipe
2025-11-30T08:31:13.786800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.786809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.786823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 51 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.786832Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.786840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.786842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.786847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.786850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.786852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.783283Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.783936Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.786202Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.787485Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 51 partNo 0
2025-11-30T08:31:13.787513Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 51 partNo 0 FormedBlobsCount 0 NewHead: Offset 51 PartNo 0 PackedSize 262214 count 1 nextOffset 52 batches 1
2025-11-30T08:31:13.787518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.787520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.787522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.787581Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 51,1 HeadOffset 51 endOffset 51 curOffset 52 d0000000000_00000000000000000051_00000_0000000001_00000? size 262204 WTime 1764491473787
2025-11-30T08:31:13.787616Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.787629Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 51 partNo 0 count 1 size 262204
2025-11-30T08:31:13.789316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 51 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.789330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.789343Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.789349Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.789356Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 51, partNo: 0, Offset: 51 is stored on disk
2025-11-30T08:31:13.789377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.789380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.789381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.789384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.789385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.789391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.789398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.789501Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 51 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.789584Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 51
    Offset: 51
    WriteTimestampMS: 1764491473786
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-30T08:31:13.789833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524874:3580] destroyed
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.790314Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.790672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524878:3582], now have 1 active actors on pipe
2025-11-30T08:31:13.790814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.790819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.790839Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|d1f0b7ed-9394675e-74b87873-aadfb047_52 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.790844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.790847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.790851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.790852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.790854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.790859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.790861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.790863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.790870Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.790877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.791016Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-30T08:31:13.791207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524878:3582] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|d1f0b7ed-9394675e-74b87873-aadfb047_52"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|d1f0b7ed-9394675e-74b87873-aadfb047_52"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 52
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.793613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524881:3583], now have 1 active actors on pipe
2025-11-30T08:31:13.793200Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.793998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.794005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.794020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 52 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.794027Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.794035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.794037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.794043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.794045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.794048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.795225Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 52 partNo 0
2025-11-30T08:31:13.795250Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 52 partNo 0 FormedBlobsCount 0 NewHead: Offset 52 PartNo 0 PackedSize 262214 count 1 nextOffset 53 batches 1
2025-11-30T08:31:13.795256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.795259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.795262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.795343Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 52,1 HeadOffset 52 endOffset 52 curOffset 53 d0000000000_00000000000000000052_00000_0000000001_00000? size 262204 WTime 1764491473795
2025-11-30T08:31:13.795379Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.795390Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 52 partNo 0 count 1 size 262204
2025-11-30T08:31:13.797409Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 52 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.797423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.797438Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.797444Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.797451Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 52, partNo: 0, Offset: 52 is stored on disk
2025-11-30T08:31:13.797475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.797477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.797479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.797481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.797484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.797489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.797497Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.797584Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 52 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.797664Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 52
    Offset: 52
    WriteTimestampMS: 1764491473793
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.798018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524881:3583] destroyed
2025-11-30T08:31:13.798461Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.798991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524885:3585], now have 1 active actors on pipe
2025-11-30T08:31:13.799000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.799004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.799022Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f88098ed-c3e14697-5f7402d5-2cf2ede0_53 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.799027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.799030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.799033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.799034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.799037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.799042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.799044Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.799047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.799055Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.799063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.799181Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f88098ed-c3e14697-5f7402d5-2cf2ede0_53"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.799454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524885:3585] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f88098ed-c3e14697-5f7402d5-2cf2ede0_53"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 53
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.801353Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.802060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524888:3586], now have 1 active actors on pipe
2025-11-30T08:31:13.802126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.802130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.802144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 53 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.802150Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.802158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.802160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.802166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.802168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.802171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.803350Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 53 partNo 0
2025-11-30T08:31:13.803376Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 53 partNo 0 FormedBlobsCount 0 NewHead: Offset 53 PartNo 0 PackedSize 262214 count 1 nextOffset 54 batches 1
2025-11-30T08:31:13.803382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.803384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.803387Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.803466Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 53,1 HeadOffset 53 endOffset 53 curOffset 54 d0000000000_00000000000000000053_00000_0000000001_00000? size 262204 WTime 1764491473802
2025-11-30T08:31:13.803506Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.803517Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 53 partNo 0 count 1 size 262204
2025-11-30T08:31:13.805539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 53 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.805553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.805569Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.805577Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.805584Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 53, partNo: 0, Offset: 53 is stored on disk
2025-11-30T08:31:13.805607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.805609Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.805611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.805613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.805615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.805620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.805629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.805787Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-30T08:31:13.805731Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 53 partno 0 count 1 parts 0 suffix '63' size 262204
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 53
    Offset: 53
    WriteTimestampMS: 1764491473801
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.806245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524888:3586] destroyed
2025-11-30T08:31:13.806713Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.807300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524892:3588], now have 1 active actors on pipe
2025-11-30T08:31:13.807309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.807314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.807335Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b4cd6e09-e2b4e0ad-f9e6c127-5661ebeb_54 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.807342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.807345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.807349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.807350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.807353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.807360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.807362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.807364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.807372Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.807380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.807505Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b4cd6e09-e2b4e0ad-f9e6c127-5661ebeb_54"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.807839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524892:3588] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b4cd6e09-e2b4e0ad-f9e6c127-5661ebeb_54"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 54
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.810022Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.810746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524895:3589], now have 1 active actors on pipe
2025-11-30T08:31:13.810814Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.810819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.810834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 54 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.810840Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.810848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.810850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.810860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.810861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.810864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.812053Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 54 partNo 0
2025-11-30T08:31:13.812080Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 54 partNo 0 FormedBlobsCount 0 NewHead: Offset 54 PartNo 0 PackedSize 262214 count 1 nextOffset 55 batches 1
2025-11-30T08:31:13.812086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.812088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.812090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.812169Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 54,1 HeadOffset 54 endOffset 54 curOffset 55 d0000000000_00000000000000000054_00000_0000000001_00000? size 262204 WTime 1764491473807
2025-11-30T08:31:13.812205Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.812216Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 54 partNo 0 count 1 size 262204
2025-11-30T08:31:13.814980Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 54 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.814995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.815011Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.815018Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.815026Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 54, partNo: 0, Offset: 54 is stored on disk
2025-11-30T08:31:13.815050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.815052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.815054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.815058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.815060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.815067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.815075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.815224Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 54
    Offset: 54
    WriteTimestampMS: 1764491473807
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.815167Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 54 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.816225Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.816808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524895:3589] destroyed
2025-11-30T08:31:13.816818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524899:3591], now have 1 active actors on pipe
2025-11-30T08:31:13.816826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.816830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.816842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.816844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.816846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.816848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.816850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.816879Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9ab756eb-1cf9d0f9-a6e3b422-e019d010_55 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.816882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.816885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.816888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.816891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.816893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.816900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.816902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.816904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.816911Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.816920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.817064Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9ab756eb-1cf9d0f9-a6e3b422-e019d010_55"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9ab756eb-1cf9d0f9-a6e3b422-e019d010_55"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 55
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.819386Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.820013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524899:3591] destroyed
2025-11-30T08:31:13.820023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524902:3592], now have 1 active actors on pipe
2025-11-30T08:31:13.820097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.820102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.820117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 55 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.820127Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.820134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.820137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.820143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.820145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.820149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.821317Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 55 partNo 0
2025-11-30T08:31:13.821342Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 55 partNo 0 FormedBlobsCount 0 NewHead: Offset 55 PartNo 0 PackedSize 262214 count 1 nextOffset 56 batches 1
2025-11-30T08:31:13.821348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.821351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.821353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.821434Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 55,1 HeadOffset 55 endOffset 55 curOffset 56 d0000000000_00000000000000000055_00000_0000000001_00000? size 262204 WTime 1764491473820
2025-11-30T08:31:13.821470Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.821481Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 55 partNo 0 count 1 size 262204
2025-11-30T08:31:13.823472Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 55 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.823485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.823500Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.823506Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.823513Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 55, partNo: 0, Offset: 55 is stored on disk
2025-11-30T08:31:13.823535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.823537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.823538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.823540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.823542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.823547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.823555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.823707Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.823655Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 55 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.824029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524902:3592] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 55
    Offset: 55
    WriteTimestampMS: 1764491473816
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 4
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.825569Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.826146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524909:3594], now have 1 active actors on pipe
2025-11-30T08:31:13.826157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.826162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.826182Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|52af3ec5-fa8150ea-956ae6bc-6f997a7e_56 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.826187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.826189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.826192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.826194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.826197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.826203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.826206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.826208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.826217Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.826225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.826342Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|52af3ec5-fa8150ea-956ae6bc-6f997a7e_56"
    Status: Active
    SeqNo: 0
  }
}
2025-11-30T08:31:13.826634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524909:3594] destroyed
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|52af3ec5-fa8150ea-956ae6bc-6f997a7e_56"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 56
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.828508Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.829111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524912:3595], now have 1 active actors on pipe
2025-11-30T08:31:13.829314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.829330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.829349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 56 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.829359Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.829373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.829377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.829383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.829385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.829388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.830426Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 56 partNo 0
2025-11-30T08:31:13.830464Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 56 partNo 0 FormedBlobsCount 0 NewHead: Offset 56 PartNo 0 PackedSize 262214 count 1 nextOffset 57 batches 1
2025-11-30T08:31:13.830477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.830484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.830487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.830590Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 56,1 HeadOffset 56 endOffset 56 curOffset 57 d0000000000_00000000000000000056_00000_0000000001_00000? size 262204 WTime 1764491473829
2025-11-30T08:31:13.830643Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.830656Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 56 partNo 0 count 1 size 262204
2025-11-30T08:31:13.832676Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 56 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.832690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.832747Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 56 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.832854Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:492: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-11-30T08:31:13.832872Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.832879Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.832886Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 56, partNo: 0, Offset: 56 is stored on disk
2025-11-30T08:31:13.832915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.832917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.832919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.832921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.832923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.832928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.832937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.833058Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 56
    Offset: 56
    WriteTimestampMS: 1764491473829
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.834045Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.834216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524912:3595] destroyed
2025-11-30T08:31:13.834593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524916:3597], now have 1 active actors on pipe
2025-11-30T08:31:13.834603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.834608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.834629Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b96602d3-1efb04a2-37501a23-2a9cce97_57 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.834635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.834638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.834641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.834643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.834646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.834652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.834654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.834656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.834665Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.834673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.834830Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b96602d3-1efb04a2-37501a23-2a9cce97_57"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.835092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524916:3597] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b96602d3-1efb04a2-37501a23-2a9cce97_57"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 57
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.836953Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.837968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524919:3598], now have 1 active actors on pipe
2025-11-30T08:31:13.838046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.838050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.838078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 57 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.838085Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.838095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.838097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.838105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.838108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.838111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.839195Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 57 partNo 0
2025-11-30T08:31:13.839231Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 57 partNo 0 FormedBlobsCount 0 NewHead: Offset 57 PartNo 0 PackedSize 262214 count 1 nextOffset 58 batches 1
2025-11-30T08:31:13.839237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.839239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.839242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.839322Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 57,1 HeadOffset 57 endOffset 57 curOffset 58 d0000000000_00000000000000000057_00000_0000000001_00000? size 262204 WTime 1764491473839
2025-11-30T08:31:13.839358Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.839370Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 57 partNo 0 count 1 size 262204
2025-11-30T08:31:13.841247Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 57 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.841263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.841277Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.841284Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.841291Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 57, partNo: 0, Offset: 57 is stored on disk
2025-11-30T08:31:13.841315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.841318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.841320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.841322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.841325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.841330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.841338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.841488Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.841433Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 57 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.841960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524919:3598] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 57
    Offset: 57
    WriteTimestampMS: 1764491473837
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.843348Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.843898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524924:3601], now have 1 active actors on pipe
2025-11-30T08:31:13.843908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.843913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.843934Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|928d67e3-d519c5af-589dc577-c5a4b604_58 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.843940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.843943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.843947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.843949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.843951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.843958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.843959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.843963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.843972Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.843981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.844097Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|928d67e3-d519c5af-589dc577-c5a4b604_58"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.844420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524924:3601] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|928d67e3-d519c5af-589dc577-c5a4b604_58"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 58
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.846300Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.847030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524927:3602], now have 1 active actors on pipe
2025-11-30T08:31:13.847101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.847106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.847122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 58 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.847128Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.847137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.847140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.847146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.847148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.847151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.848465Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 58 partNo 0
2025-11-30T08:31:13.848492Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 58 partNo 0 FormedBlobsCount 0 NewHead: Offset 58 PartNo 0 PackedSize 262214 count 1 nextOffset 59 batches 1
2025-11-30T08:31:13.848497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.848499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.848503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.848582Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 58,1 HeadOffset 58 endOffset 58 curOffset 59 d0000000000_00000000000000000058_00000_0000000001_00000? size 262204 WTime 1764491473848
2025-11-30T08:31:13.848621Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.848631Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 58 partNo 0 count 1 size 262204
2025-11-30T08:31:13.850684Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 58 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.850698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.850712Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.850718Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.850726Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 58, partNo: 0, Offset: 58 is stored on disk
2025-11-30T08:31:13.850749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.850751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.850753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.850757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.850758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.850763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.850770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.850917Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 58
    Offset: 58
    WriteTimestampMS: 1764491473846
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.850863Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 58 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.851469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524927:3602] destroyed
2025-11-30T08:31:13.851845Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.852289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524931:3604], now have 1 active actors on pipe
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|3dca6fac-d1ae8216-930a243-32eca4ec_59"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|3dca6fac-d1ae8216-930a243-32eca4ec_59"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 59
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.852602Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.852388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.852399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.852450Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|3dca6fac-d1ae8216-930a243-32eca4ec_59 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.852461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.852468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.852474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.852476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.852480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.852491Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.852494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.852497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.852514Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.852532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.854948Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.855040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524931:3604] destroyed
2025-11-30T08:31:13.855880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524934:3605], now have 1 active actors on pipe
2025-11-30T08:31:13.856023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.856084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.856115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 59 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.856127Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.856144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.856151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.856162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.856163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.856166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.856909Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 59 partNo 0
2025-11-30T08:31:13.856958Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 59 partNo 0 FormedBlobsCount 0 NewHead: Offset 59 PartNo 0 PackedSize 262214 count 1 nextOffset 60 batches 1
2025-11-30T08:31:13.856972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.856979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.856982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.857072Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 59,1 HeadOffset 59 endOffset 59 curOffset 60 d0000000000_00000000000000000059_00000_0000000001_00000? size 262204 WTime 1764491473856
2025-11-30T08:31:13.857116Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.857125Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 59 partNo 0 count 1 size 262204
2025-11-30T08:31:13.859408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 59 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.859431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.859451Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.859459Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.859468Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 59, partNo: 0, Offset: 59 is stored on disk
2025-11-30T08:31:13.859496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.859631Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.859499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.859501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.859505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.859507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.859514Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.859524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.859572Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 59 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.859925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524934:3605] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 59
    Offset: 59
    WriteTimestampMS: 1764491473855
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.860466Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.860854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524938:3607], now have 1 active actors on pipe
2025-11-30T08:31:13.861027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.861035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.861059Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9b122b17-4e4384a8-50cde6d9-ecccada3_60 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.861067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.861070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.861073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.861075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.861078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.861085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.861088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.861090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.861100Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.861110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.861180Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9b122b17-4e4384a8-50cde6d9-ecccada3_60"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.861743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524938:3607] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9b122b17-4e4384a8-50cde6d9-ecccada3_60"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 60
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.863550Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.864339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524941:3608], now have 1 active actors on pipe
2025-11-30T08:31:13.864411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.864416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.864435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 60 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.864442Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.864452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.864455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.864463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.864465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.864468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.865675Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 60 partNo 0
2025-11-30T08:31:13.865702Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 60 partNo 0 FormedBlobsCount 0 NewHead: Offset 60 PartNo 0 PackedSize 262214 count 1 nextOffset 61 batches 1
2025-11-30T08:31:13.865708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.865711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.865713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.865793Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 60,1 HeadOffset 60 endOffset 60 curOffset 61 d0000000000_00000000000000000060_00000_0000000001_00000? size 262204 WTime 1764491473865
2025-11-30T08:31:13.865841Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.865856Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 60 partNo 0 count 1 size 262204
2025-11-30T08:31:13.867892Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 60 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.867910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.867925Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.867933Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.867942Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 60, partNo: 0, Offset: 60 is stored on disk
2025-11-30T08:31:13.867969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.867977Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.867978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.867980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.867982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.867988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.867997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.868047Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 60 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.868130Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 60
    Offset: 60
    WriteTimestampMS: 1764491473864
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.869015Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.868733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524941:3608] destroyed
2025-11-30T08:31:13.869722Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|7196ac15-10cd59d8-f8f0ee37-53c1e83a_61"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:13.869576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524945:3610], now have 1 active actors on pipe
2025-11-30T08:31:13.869585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.869591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.869611Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|7196ac15-10cd59d8-f8f0ee37-53c1e83a_61 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.869617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.869620Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.869623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.869624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.869627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.869632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.869634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.869636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.869644Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.869651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|7196ac15-10cd59d8-f8f0ee37-53c1e83a_61"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 61
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.870971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524945:3610] destroyed
2025-11-30T08:31:13.872002Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.872404Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524948:3611], now have 1 active actors on pipe
2025-11-30T08:31:13.872782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.872815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.872835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 61 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.872845Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.872856Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.872860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.872869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.872871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.872875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.873834Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 61 partNo 0
2025-11-30T08:31:13.873877Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 61 partNo 0 FormedBlobsCount 0 NewHead: Offset 61 PartNo 0 PackedSize 262214 count 1 nextOffset 62 batches 1
2025-11-30T08:31:13.873889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.873891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.873894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.873984Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 61,1 HeadOffset 61 endOffset 61 curOffset 62 d0000000000_00000000000000000061_00000_0000000001_00000? size 262204 WTime 1764491473873
2025-11-30T08:31:13.874033Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.874050Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 61 partNo 0 count 1 size 262204
2025-11-30T08:31:13.875908Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 61 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.875932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.875948Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.875951Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 61 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.875960Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.875969Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 61, partNo: 0, Offset: 61 is stored on disk
2025-11-30T08:31:13.875995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.876005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.876008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.876010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.876012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.876017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.876028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.876092Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 61
    Offset: 61
    WriteTimestampMS: 1764491473872
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.876590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524948:3611] destroyed
2025-11-30T08:31:13.876863Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.877194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524953:3614], now have 1 active actors on pipe
2025-11-30T08:31:13.877264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.877271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.877300Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ea58f957-c451b59a-a866ceb5-b30b06f0_62 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.877306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.877309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.877312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.877314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.877317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.877323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.877325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.877327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.877337Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.877347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.877398Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.877571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524953:3614] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ea58f957-c451b59a-a866ceb5-b30b06f0_62"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ea58f957-c451b59a-a866ceb5-b30b06f0_62"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 62
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.879780Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.880474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524956:3615], now have 1 active actors on pipe
2025-11-30T08:31:13.880546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.880551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.880568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 62 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.880575Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.880587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.880589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.880597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.880599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.880602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.881780Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 62 partNo 0
2025-11-30T08:31:13.881803Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 62 partNo 0 FormedBlobsCount 0 NewHead: Offset 62 PartNo 0 PackedSize 262214 count 1 nextOffset 63 batches 1
2025-11-30T08:31:13.881809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.881812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.881814Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.881897Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 62,1 HeadOffset 62 endOffset 62 curOffset 63 d0000000000_00000000000000000062_00000_0000000001_00000? size 262204 WTime 1764491473881
2025-11-30T08:31:13.881933Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.881944Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 62 partNo 0 count 1 size 262204
2025-11-30T08:31:13.883931Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 62 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.883953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.883969Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.883980Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.883981Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.883989Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 62, partNo: 0, Offset: 62 is stored on disk
2025-11-30T08:31:13.884016Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.884019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.884021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.884023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.884025Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.884030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.884040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.884103Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.884471Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524956:3615] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 62
    Offset: 62
    WriteTimestampMS: 1764491473880
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.885356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524960:3617], now have 1 active actors on pipe
2025-11-30T08:31:13.885493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.885516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.885550Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ef28daed-13a04693-9dbe8077-88c95f95_63 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.885560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.885562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.885566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.885568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.885570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.885577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.885579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.885582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.885590Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.885599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.885959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524960:3617] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ef28daed-13a04693-9dbe8077-88c95f95_63"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ef28daed-13a04693-9dbe8077-88c95f95_63"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 63
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.888282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524963:3618], now have 1 active actors on pipe
2025-11-30T08:31:13.888654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.888667Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.888684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 63 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.888696Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.888707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.888714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.888721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.888724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.888727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.885032Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.885665Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.888000Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.889743Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 63 partNo 0
2025-11-30T08:31:13.889781Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 63 partNo 0 FormedBlobsCount 0 NewHead: Offset 63 PartNo 0 PackedSize 262214 count 1 nextOffset 64 batches 1
2025-11-30T08:31:13.889793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.889795Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.889798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.889889Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 63,1 HeadOffset 63 endOffset 63 curOffset 64 d0000000000_00000000000000000063_00000_0000000001_00000? size 262204 WTime 1764491473889
2025-11-30T08:31:13.889940Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.889956Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 63 partNo 0 count 1 size 262204
2025-11-30T08:31:13.891817Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 63 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.891844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.891869Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.891876Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.891885Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 63, partNo: 0, Offset: 63 is stored on disk
2025-11-30T08:31:13.891906Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 63 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.891917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.891920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.891922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.891925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.891927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.891934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000032_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.891937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000033_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.891941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000034_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.891944Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000035_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.891947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000036_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.891950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000037_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.891952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000038_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.891955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000039_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.891957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000040_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.891960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000041_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.891963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000042_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.891966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000043_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.891969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000044_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.891972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000045_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.891975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000046_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.891978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000047_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.891981Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000048_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.891983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000049_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.891986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000050_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.891989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000051_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.891992Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000052_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.891995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000053_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.891998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000054_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.892000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000055_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.892003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000056_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.892006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000057_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.892008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000058_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.892021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000059_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.892025Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000060_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.892027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000061_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.892029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000062_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.892032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000063_00000_0000000001_00000? (262204)
2025-11-30T08:31:13.892034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 32 keys were taken away. Let's read 8390528 bytes
2025-11-30T08:31:13.892045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.892062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 32 blobs
2025-11-30T08:31:13.892069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000032_00000_0000000001_00000?
2025-11-30T08:31:13.892078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000033_00000_0000000001_00000?
2025-11-30T08:31:13.892089Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000034_00000_0000000001_00000?
2025-11-30T08:31:13.892100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000035_00000_0000000001_00000?
2025-11-30T08:31:13.892112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000036_00000_0000000001_00000?
2025-11-30T08:31:13.892130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000037_00000_0000000001_00000?
2025-11-30T08:31:13.892138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000038_00000_0000000001_00000?
2025-11-30T08:31:13.892147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000039_00000_0000000001_00000?
2025-11-30T08:31:13.892157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000040_00000_0000000001_00000?
2025-11-30T08:31:13.892181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000041_00000_0000000001_00000?
2025-11-30T08:31:13.892190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000042_00000_0000000001_00000?
2025-11-30T08:31:13.892198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000043_00000_0000000001_00000?
2025-11-30T08:31:13.892206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000044_00000_0000000001_00000?
2025-11-30T08:31:13.892215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000045_00000_0000000001_00000?
2025-11-30T08:31:13.892223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000046_00000_0000000001_00000?
2025-11-30T08:31:13.892232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000047_00000_0000000001_00000?
2025-11-30T08:31:13.892242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000048_00000_0000000001_00000?
2025-11-30T08:31:13.892283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000049_00000_0000000001_00000?
2025-11-30T08:31:13.892292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000050_00000_0000000001_00000?
2025-11-30T08:31:13.892301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000051_00000_0000000001_00000?
2025-11-30T08:31:13.892310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000052_00000_0000000001_00000?
2025-11-30T08:31:13.892318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000053_00000_0000000001_00000?
2025-11-30T08:31:13.892327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000054_00000_0000000001_00000?
2025-11-30T08:31:13.892336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000055_00000_0000000001_00000?
2025-11-30T08:31:13.892343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000056_00000_0000000001_00000?
2025-11-30T08:31:13.892352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000057_00000_0000000001_00000?
2025-11-30T08:31:13.892361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000058_00000_0000000001_00000?
2025-11-30T08:31:13.892379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000059_00000_0000000001_00000?
2025-11-30T08:31:13.892389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000060_00000_0000000001_00000?
2025-11-30T08:31:13.892397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000061_00000_0000000001_00000?
2025-11-30T08:31:13.892406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000062_00000_0000000001_00000?
2025-11-30T08:31:13.892414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000063_00000_0000000001_00000?
2025-11-30T08:31:13.892424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 32 blobs for compaction
2025-11-30T08:31:13.892434Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 32 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892437Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 33 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892441Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 34 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892444Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 35 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892447Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 36 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892451Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 37 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892455Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 38 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892458Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 39 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892461Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 40 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892463Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 41 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892466Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 42 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892469Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 43 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892473Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 44 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892476Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 45 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892480Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 46 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892483Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 47 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892486Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 48 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892492Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 49 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892496Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 50 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892499Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 51 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892502Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 52 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 53 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892509Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 54 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892512Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 55 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 56 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892519Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 57 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892522Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 58 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 59 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892531Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 60 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892534Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 61 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892537Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 62 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 63 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:13.892554Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 32 blobs are from cache.
2025-11-30T08:31:13.892587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-30T08:31:13.892593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/32] d0000000000_00000000000000000032_00000_0000000001_00000?
2025-11-30T08:31:13.892595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.892598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000032_00000_0000000001_00000?, parameters.CurOffset=32
2025-11-30T08:31:13.892682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 32.0/1
2025-11-30T08:31:13.892132Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.892689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 32 partNo 0
2025-11-30T08:31:13.892711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 32 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 262214 count 1 nextOffset 33 batches 1
2025-11-30T08:31:13.892714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 32:0 appended
2025-11-30T08:31:13.892718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/32] d0000000000_00000000000000000033_00000_0000000001_00000?
2025-11-30T08:31:13.892720Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 32 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.892724Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 33 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000033_00000_0000000001_00000?, parameters.CurOffset=33
2025-11-30T08:31:13.892727Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 34 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892731Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 35 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892734Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 36 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892737Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 37 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892740Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 38 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892744Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 39 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892747Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 40 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892750Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 41 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892753Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 42 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892755Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 43 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892758Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 44 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892761Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 45 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892764Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 46 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892767Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 47 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892769Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 48 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892774Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 49 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892777Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 50 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892779Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 51 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892782Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 52 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892784Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 53 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892800Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 54 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892803Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 55 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892806Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 56 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892808Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 57 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892811Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 58 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892813Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 59 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892816Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 60 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 33.0/1
2025-11-30T08:31:13.892819Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 61 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892821Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:13.892822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 33 partNo 0
2025-11-30T08:31:13.892824Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 63 partno 0 count 1 parts 0 suffix '63'
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 63
    Offset: 63
    WriteTimestampMS: 1764491473888
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.893132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 33 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 524387 count 2 nextOffset 34 batches 1
2025-11-30T08:31:13.893146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 33:0 appended
2025-11-30T08:31:13.893151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/32] d0000000000_00000000000000000034_00000_0000000001_00000?
2025-11-30T08:31:13.893152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.893155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000034_00000_0000000001_00000?, parameters.CurOffset=34
2025-11-30T08:31:13.893227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 34.0/1
2025-11-30T08:31:13.893235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 34 partNo 0
2025-11-30T08:31:13.893246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 34 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 786601 count 3 nextOffset 35 batches 2
2025-11-30T08:31:13.893249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 34:0 appended
2025-11-30T08:31:13.893252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/32] d0000000000_00000000000000000035_00000_0000000001_00000?
2025-11-30T08:31:13.893254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.893256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000035_00000_0000000001_00000?, parameters.CurOffset=35
2025-11-30T08:31:13.893314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 35.0/1
2025-11-30T08:31:13.893323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 35 partNo 0
2025-11-30T08:31:13.893777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 35 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 1048774 count 4 nextOffset 36 batches 2
2025-11-30T08:31:13.893787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 35:0 appended
2025-11-30T08:31:13.893791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/32] d0000000000_00000000000000000036_00000_0000000001_00000?
2025-11-30T08:31:13.893793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.893796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000036_00000_0000000001_00000?, parameters.CurOffset=36
2025-11-30T08:31:13.893861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 36.0/1
2025-11-30T08:31:13.893865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 36 partNo 0
2025-11-30T08:31:13.893875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 36 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 1310988 count 5 nextOffset 37 batches 3
2025-11-30T08:31:13.893882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 36:0 appended
2025-11-30T08:31:13.893885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/32] d0000000000_00000000000000000037_00000_0000000001_00000?
2025-11-30T08:31:13.893887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.893889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000037_00000_0000000001_00000?, parameters.CurOffset=37
2025-11-30T08:31:13.893948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 37.0/1
2025-11-30T08:31:13.893952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 37 partNo 0
2025-11-30T08:31:13.894418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 37 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 1573161 count 6 nextOffset 38 batches 3
2025-11-30T08:31:13.894432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 37:0 appended
2025-11-30T08:31:13.894436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/32] d0000000000_00000000000000000038_00000_0000000001_00000?
2025-11-30T08:31:13.894438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.894441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000038_00000_0000000001_00000?, parameters.CurOffset=38
2025-11-30T08:31:13.894500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 38.0/1
2025-11-30T08:31:13.894510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 38 partNo 0
2025-11-30T08:31:13.894521Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 38 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 1835375 count 7 nextOffset 39 batches 4
2025-11-30T08:31:13.894524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 38:0 appended
2025-11-30T08:31:13.894528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/32] d0000000000_00000000000000000039_00000_0000000001_00000?
2025-11-30T08:31:13.894529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.894532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000039_00000_0000000001_00000?, parameters.CurOffset=39
2025-11-30T08:31:13.894586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 39.0/1
2025-11-30T08:31:13.894595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 39 partNo 0
2025-11-30T08:31:13.895986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 39 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 2097548 count 8 nextOffset 40 batches 4
2025-11-30T08:31:13.895999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 39:0 appended
2025-11-30T08:31:13.896004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[8/32] d0000000000_00000000000000000040_00000_0000000001_00000?
2025-11-30T08:31:13.896006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.896009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000040_00000_0000000001_00000?, parameters.CurOffset=40
2025-11-30T08:31:13.893567Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.896080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 40.0/1
2025-11-30T08:31:13.896091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 40 partNo 0
2025-11-30T08:31:13.896101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 40 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 2359762 count 9 nextOffset 41 batches 5
2025-11-30T08:31:13.896103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 40:0 appended
2025-11-30T08:31:13.896107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[9/32] d0000000000_00000000000000000041_00000_0000000001_00000?
2025-11-30T08:31:13.896109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.896113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000041_00000_0000000001_00000?, parameters.CurOffset=41
2025-11-30T08:31:13.896154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 41.0/1
2025-11-30T08:31:13.896164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 41 partNo 0
2025-11-30T08:31:13.896360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 41 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 2621935 count 10 nextOffset 42 batches 5
2025-11-30T08:31:13.896372Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 41:0 appended
2025-11-30T08:31:13.896377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[10/32] d0000000000_00000000000000000042_00000_0000000001_00000?
2025-11-30T08:31:13.896379Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.896381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000042_00000_0000000001_00000?, parameters.CurOffset=42
2025-11-30T08:31:13.896445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 42.0/1
2025-11-30T08:31:13.896454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 42 partNo 0
2025-11-30T08:31:13.896490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 42 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 2884149 count 11 nextOffset 43 batches 6
2025-11-30T08:31:13.896497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 42:0 appended
2025-11-30T08:31:13.896502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[11/32] d0000000000_00000000000000000043_00000_0000000001_00000?
2025-11-30T08:31:13.896504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.896506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000043_00000_0000000001_00000?, parameters.CurOffset=43
2025-11-30T08:31:13.896546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 43.0/1
2025-11-30T08:31:13.896554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 43 partNo 0
2025-11-30T08:31:13.896974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 43 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 3146322 count 12 nextOffset 44 batches 6
2025-11-30T08:31:13.896985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 43:0 appended
2025-11-30T08:31:13.896990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[12/32] d0000000000_00000000000000000044_00000_0000000001_00000?
2025-11-30T08:31:13.896992Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.896995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000044_00000_0000000001_00000?, parameters.CurOffset=44
2025-11-30T08:31:13.897044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 44.0/1
2025-11-30T08:31:13.897053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 44 partNo 0
2025-11-30T08:31:13.897064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 44 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 3408536 count 13 nextOffset 45 batches 7
2025-11-30T08:31:13.897067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 44:0 appended
2025-11-30T08:31:13.897071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[13/32] d0000000000_00000000000000000045_00000_0000000001_00000?
2025-11-30T08:31:13.897073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.897075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000045_00000_0000000001_00000?, parameters.CurOffset=45
2025-11-30T08:31:13.897119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 45.0/1
2025-11-30T08:31:13.897127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 45 partNo 0
2025-11-30T08:31:13.897246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 45 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 3670709 count 14 nextOffset 46 batches 7
2025-11-30T08:31:13.897255Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 45:0 appended
2025-11-30T08:31:13.897260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[14/32] d0000000000_00000000000000000046_00000_0000000001_00000?
2025-11-30T08:31:13.897263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.897266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000046_00000_0000000001_00000?, parameters.CurOffset=46
2025-11-30T08:31:13.897317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 46.0/1
2025-11-30T08:31:13.897327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 46 partNo 0
2025-11-30T08:31:13.897336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 46 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 3932923 count 15 nextOffset 47 batches 8
2025-11-30T08:31:13.897342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 46:0 appended
2025-11-30T08:31:13.897345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[15/32] d0000000000_00000000000000000047_00000_0000000001_00000?
2025-11-30T08:31:13.897347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.897350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000047_00000_0000000001_00000?, parameters.CurOffset=47
2025-11-30T08:31:13.897393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 47.0/1
2025-11-30T08:31:13.897402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 47 partNo 0
2025-11-30T08:31:13.897518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 47 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 4195096 count 16 nextOffset 48 batches 8
2025-11-30T08:31:13.897526Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 47:0 appended
2025-11-30T08:31:13.897531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[16/32] d0000000000_00000000000000000048_00000_0000000001_00000?
2025-11-30T08:31:13.897533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.897535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000048_00000_0000000001_00000?, parameters.CurOffset=48
2025-11-30T08:31:13.897583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 48.0/1
2025-11-30T08:31:13.897591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 48 partNo 0
2025-11-30T08:31:13.897600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 48 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 4457310 count 17 nextOffset 49 batches 9
2025-11-30T08:31:13.897604Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 48:0 appended
2025-11-30T08:31:13.897607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[17/32] d0000000000_00000000000000000049_00000_0000000001_00000?
2025-11-30T08:31:13.897608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.897611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000049_00000_0000000001_00000?, parameters.CurOffset=49
2025-11-30T08:31:13.897652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 49.0/1
2025-11-30T08:31:13.897660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 49 partNo 0
2025-11-30T08:31:13.897774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 49 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 4719483 count 18 nextOffset 50 batches 9
2025-11-30T08:31:13.897783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 49:0 appended
2025-11-30T08:31:13.897787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[18/32] d0000000000_00000000000000000050_00000_0000000001_00000?
2025-11-30T08:31:13.897789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.897791Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000050_00000_0000000001_00000?, parameters.CurOffset=50
2025-11-30T08:31:13.897840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 50.0/1
2025-11-30T08:31:13.897850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 50 partNo 0
2025-11-30T08:31:13.897858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 50 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 4981697 count 19 nextOffset 51 batches 10
2025-11-30T08:31:13.897862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 50:0 appended
2025-11-30T08:31:13.897865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[19/32] d0000000000_00000000000000000051_00000_0000000001_00000?
2025-11-30T08:31:13.897867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.897869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000051_00000_0000000001_00000?, parameters.CurOffset=51
2025-11-30T08:31:13.897911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 51.0/1
2025-11-30T08:31:13.897919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 51 partNo 0
2025-11-30T08:31:13.898036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 51 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 5243870 count 20 nextOffset 52 batches 10
2025-11-30T08:31:13.898046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 51:0 appended
2025-11-30T08:31:13.898050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[20/32] d0000000000_00000000000000000052_00000_0000000001_00000?
2025-11-30T08:31:13.898053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.898055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000052_00000_0000000001_00000?, parameters.CurOffset=52
2025-11-30T08:31:13.898107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 52.0/1
2025-11-30T08:31:13.898115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 52 partNo 0
2025-11-30T08:31:13.898126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 52 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 5506084 count 21 nextOffset 53 batches 11
2025-11-30T08:31:13.898130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 52:0 appended
2025-11-30T08:31:13.898133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[21/32] d0000000000_00000000000000000053_00000_0000000001_00000?
2025-11-30T08:31:13.898135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.898137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000053_00000_0000000001_00000?, parameters.CurOffset=53
2025-11-30T08:31:13.898180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 53.0/1
2025-11-30T08:31:13.898189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 53 partNo 0
2025-11-30T08:31:13.898305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 53 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 5768257 count 22 nextOffset 54 batches 11
2025-11-30T08:31:13.898314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 53:0 appended
2025-11-30T08:31:13.898318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[22/32] d0000000000_00000000000000000054_00000_0000000001_00000?
2025-11-30T08:31:13.898320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.898322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000054_00000_0000000001_00000?, parameters.CurOffset=54
2025-11-30T08:31:13.898371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 54.0/1
2025-11-30T08:31:13.898380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 54 partNo 0
2025-11-30T08:31:13.898389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 54 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 6030471 count 23 nextOffset 55 batches 12
2025-11-30T08:31:13.898393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 54:0 appended
2025-11-30T08:31:13.898396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[23/32] d0000000000_00000000000000000055_00000_0000000001_00000?
2025-11-30T08:31:13.898397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.898400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000055_00000_0000000001_00000?, parameters.CurOffset=55
2025-11-30T08:31:13.898441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 55.0/1
2025-11-30T08:31:13.898449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 55 partNo 0
2025-11-30T08:31:13.898561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 55 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 6292644 count 24 nextOffset 56 batches 12
2025-11-30T08:31:13.898569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 55:0 appended
2025-11-30T08:31:13.898573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[24/32] d0000000000_00000000000000000056_00000_0000000001_00000?
2025-11-30T08:31:13.898575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.898577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000056_00000_0000000001_00000?, parameters.CurOffset=56
2025-11-30T08:31:13.898624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 56.0/1
2025-11-30T08:31:13.898632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 56 partNo 0
2025-11-30T08:31:13.898642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 56 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 6554858 count 25 nextOffset 57 batches 13
2025-11-30T08:31:13.898645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 56:0 appended
2025-11-30T08:31:13.898649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[25/32] d0000000000_00000000000000000057_00000_0000000001_00000?
2025-11-30T08:31:13.898650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.898652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000057_00000_0000000001_00000?, parameters.CurOffset=57
2025-11-30T08:31:13.898692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 57.0/1
2025-11-30T08:31:13.898699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 57 partNo 0
2025-11-30T08:31:13.899164Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 57 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 6817031 count 26 nextOffset 58 batches 13
2025-11-30T08:31:13.899176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 57:0 appended
2025-11-30T08:31:13.899181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[26/32] d0000000000_00000000000000000058_00000_0000000001_00000?
2025-11-30T08:31:13.899182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.899185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000058_00000_0000000001_00000?, parameters.CurOffset=58
2025-11-30T08:31:13.899234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 58.0/1
2025-11-30T08:31:13.899243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 58 partNo 0
2025-11-30T08:31:13.899263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 58 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 7079245 count 27 nextOffset 59 batches 14
2025-11-30T08:31:13.899269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 58:0 appended
2025-11-30T08:31:13.899272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[27/32] d0000000000_00000000000000000059_00000_0000000001_00000?
2025-11-30T08:31:13.899274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.899276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000059_00000_0000000001_00000?, parameters.CurOffset=59
2025-11-30T08:31:13.899325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 59.0/1
2025-11-30T08:31:13.899333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 59 partNo 0
2025-11-30T08:31:13.899802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 59 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 7341418 count 28 nextOffset 60 batches 14
2025-11-30T08:31:13.899816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 59:0 appended
2025-11-30T08:31:13.899822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[28/32] d0000000000_00000000000000000060_00000_0000000001_00000?
2025-11-30T08:31:13.899824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.899826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000060_00000_0000000001_00000?, parameters.CurOffset=60
2025-11-30T08:31:13.899890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 60.0/1
2025-11-30T08:31:13.899900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 60 partNo 0
2025-11-30T08:31:13.899911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 60 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 7603632 count 29 nextOffset 61 batches 15
2025-11-30T08:31:13.899915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 60:0 appended
2025-11-30T08:31:13.899918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[29/32] d0000000000_00000000000000000061_00000_0000000001_00000?
2025-11-30T08:31:13.899920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.899923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000061_00000_0000000001_00000?, parameters.CurOffset=61
2025-11-30T08:31:13.899964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 61.0/1
2025-11-30T08:31:13.899972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 61 partNo 0
2025-11-30T08:31:13.900432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 61 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 7865805 count 30 nextOffset 62 batches 15
2025-11-30T08:31:13.900444Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 61:0 appended
2025-11-30T08:31:13.900449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[30/32] d0000000000_00000000000000000062_00000_0000000001_00000?
2025-11-30T08:31:13.900452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.900455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000062_00000_0000000001_00000?, parameters.CurOffset=62
2025-11-30T08:31:13.900503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 62.0/1
2025-11-30T08:31:13.900514Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 62 partNo 0
2025-11-30T08:31:13.906990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 62 partNo 0 result is x0000000000_00000000000000000031_00000_0000000031_00000 size 8128009
2025-11-30T08:31:13.907017Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1106: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000031_00000_0000000031_00000 new key d0000000000_00000000000000000031_00000_0000000031_00000 size 8128009 WTime 1764491473906
2025-11-30T08:31:13.907047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 62 partNo 0 FormedBlobsCount 1 NewHead: Offset 62 PartNo 0 PackedSize 262214 count 1 nextOffset 63 batches 1
2025-11-30T08:31:13.907053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 62:0 appended
2025-11-30T08:31:13.907058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[31/32] d0000000000_00000000000000000063_00000_0000000001_00000?
2025-11-30T08:31:13.907061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:13.907065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000063_00000_0000000001_00000?, parameters.CurOffset=63
2025-11-30T08:31:13.907126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 63.0/1
2025-11-30T08:31:13.907137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 63 partNo 0
2025-11-30T08:31:13.907269Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 63 partNo 0 FormedBlobsCount 0 NewHead: Offset 62 PartNo 0 PackedSize 524387 count 2 nextOffset 64 batches 1
2025-11-30T08:31:13.907280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 63:0 appended
2025-11-30T08:31:13.907329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:636: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 62,2 HeadOffset 31 endOffset 32 curOffset 64 d0000000000_00000000000000000062_00000_0000000002_00000| size 524387 WTime 1764491473906
2025-11-30T08:31:13.907494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524963:3618] destroyed
2025-11-30T08:31:13.907509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524967:3620], now have 1 active actors on pipe
2025-11-30T08:31:13.907531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.907536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.907551Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.907562Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 31 partNo 0 count 31 size 8128009
2025-11-30T08:31:13.907568Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 62 partNo 0 count 2 size 524387
2025-11-30T08:31:13.907603Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|dcfdbd1-4841f8d9-bdfd8f3-a861f331_64 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.907611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.907613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.907618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.907621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.907624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.907634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.907636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.907639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.907652Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.907699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.907984Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|dcfdbd1-4841f8d9-bdfd8f3-a861f331_64"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|dcfdbd1-4841f8d9-bdfd8f3-a861f331_64"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 64
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.908560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524967:3620] destroyed
2025-11-30T08:31:13.911798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524973:3622], now have 1 active actors on pipe
2025-11-30T08:31:13.912118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.912125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.912143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 64 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.912153Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.912166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.912169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.912177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.912179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.912183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.910652Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.913244Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 64 partNo 0
2025-11-30T08:31:13.913276Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 64 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 262214 count 1 nextOffset 65 batches 1
2025-11-30T08:31:13.913284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.913287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.913290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.913375Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 64,1 HeadOffset 64 endOffset 64 curOffset 65 d0000000000_00000000000000000064_00000_0000000001_00000? size 262204 WTime 1764491473913
2025-11-30T08:31:13.913415Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.913425Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 64 partNo 0 count 1 size 262204
2025-11-30T08:31:13.917231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2279: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-30T08:31:13.923521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 31 count 31 size 8128009 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.923539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 62 count 2 size 524387 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.923555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.923566Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 31 parts 0 suffix '0' size 8128009
2025-11-30T08:31:13.923575Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 2 parts 0 suffix '124' size 524387
2025-11-30T08:31:13.923870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2279: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-30T08:31:13.923878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.924037Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 64 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.924053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.924069Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 64 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.924073Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.924083Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.924093Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 64, partNo: 0, Offset: 64 is stored on disk
2025-11-30T08:31:13.924116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.924122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.924124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.924127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.924129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.924135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000031_00000_0000000001_00000|
2025-11-30T08:31:13.924139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000032_00000_0000000001_00000?
2025-11-30T08:31:13.924141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000033_00000_0000000001_00000?
2025-11-30T08:31:13.924144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000034_00000_0000000001_00000?
2025-11-30T08:31:13.924147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000035_00000_0000000001_00000?
2025-11-30T08:31:13.924149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000036_00000_0000000001_00000?
2025-11-30T08:31:13.924152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000037_00000_0000000001_00000?
2025-11-30T08:31:13.924155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000038_00000_0000000001_00000?
2025-11-30T08:31:13.924159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000039_00000_0000000001_00000?
2025-11-30T08:31:13.924161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000040_00000_0000000001_00000?
2025-11-30T08:31:13.924164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000041_00000_0000000001_00000?
2025-11-30T08:31:13.924166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000042_00000_0000000001_00000?
2025-11-30T08:31:13.924177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000043_00000_0000000001_00000?
2025-11-30T08:31:13.924180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000044_00000_0000000001_00000?
2025-11-30T08:31:13.924184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000045_00000_0000000001_00000?
2025-11-30T08:31:13.924186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000046_00000_0000000001_00000?
2025-11-30T08:31:13.924189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000047_00000_0000000001_00000?
2025-11-30T08:31:13.924192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000048_00000_0000000001_00000?
2025-11-30T08:31:13.924194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000049_00000_0000000001_00000?
2025-11-30T08:31:13.924197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000050_00000_0000000001_00000?
2025-11-30T08:31:13.924199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000051_00000_0000000001_00000?
2025-11-30T08:31:13.924201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000052_00000_0000000001_00000?
2025-11-30T08:31:13.924204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000053_00000_0000000001_00000?
2025-11-30T08:31:13.924206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000054_00000_0000000001_00000?
2025-11-30T08:31:13.924208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000055_00000_0000000001_00000?
2025-11-30T08:31:13.924210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000056_00000_0000000001_00000?
2025-11-30T08:31:13.924212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000057_00000_0000000001_00000?
2025-11-30T08:31:13.924214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000058_00000_0000000001_00000?
2025-11-30T08:31:13.924216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000059_00000_0000000001_00000?
2025-11-30T08:31:13.924218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000060_00000_0000000001_00000?
2025-11-30T08:31:13.924220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000061_00000_0000000001_00000?
2025-11-30T08:31:13.924221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000062_00000_0000000001_00000?
2025-11-30T08:31:13.924224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000063_00000_0000000001_00000?
2025-11-30T08:31:13.924242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.924252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.924269Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.924278Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000031_00000_0000000001_00000|(+) to d0000000000_00000000000000000031_00000_0000000001_00000|(+)
2025-11-30T08:31:13.924280Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000032_00000_0000000001_00000?(+) to d0000000000_00000000000000000032_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924282Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000033_00000_0000000001_00000?(+) to d0000000000_00000000000000000033_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924284Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000034_00000_0000000001_00000?(+) to d0000000000_00000000000000000034_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924286Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000035_00000_0000000001_00000?(+) to d0000000000_00000000000000000035_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924288Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000036_00000_0000000001_00000?(+) to d0000000000_00000000000000000036_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924291Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000037_00000_0000000001_00000?(+) to d0000000000_00000000000000000037_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924293Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000038_00000_0000000001_00000?(+) to d0000000000_00000000000000000038_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924295Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000039_00000_0000000001_00000?(+) to d0000000000_00000000000000000039_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924297Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000040_00000_0000000001_00000?(+) to d0000000000_00000000000000000040_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924299Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000041_00000_0000000001_00000?(+) to d0000000000_00000000000000000041_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924301Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000042_00000_0000000001_00000?(+) to d0000000000_00000000000000000042_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924303Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000043_00000_0000000001_00000?(+) to d0000000000_00000000000000000043_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924305Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000044_00000_0000000001_00000?(+) to d0000000000_00000000000000000044_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924306Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000045_00000_0000000001_00000?(+) to d0000000000_00000000000000000045_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924308Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000046_00000_0000000001_00000?(+) to d0000000000_00000000000000000046_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924309Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000047_00000_0000000001_00000?(+) to d0000000000_00000000000000000047_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924311Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000048_00000_0000000001_00000?(+) to d0000000000_00000000000000000048_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924312Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000049_00000_0000000001_00000?(+) to d0000000000_00000000000000000049_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924314Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000050_00000_0000000001_00000?(+) to d0000000000_00000000000000000050_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924315Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000051_00000_0000000001_00000?(+) to d0000000000_00000000000000000051_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924317Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000052_00000_0000000001_00000?(+) to d0000000000_00000000000000000052_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924318Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000053_00000_0000000001_00000?(+) to d0000000000_00000000000000000053_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924320Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000054_00000_0000000001_00000?(+) to d0000000000_00000000000000000054_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924322Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000055_00000_0000000001_00000?(+) to d0000000000_00000000000000000055_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924323Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000056_00000_0000000001_00000?(+) to d0000000000_00000000000000000056_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924325Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000057_00000_0000000001_00000?(+) to d0000000000_00000000000000000057_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924326Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000058_00000_0000000001_00000?(+) to d0000000000_00000000000000000058_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924328Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000059_00000_0000000001_00000?(+) to d0000000000_00000000000000000059_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924329Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000060_00000_0000000001_00000?(+) to d0000000000_00000000000000000060_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924331Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000061_00000_0000000001_00000?(+) to d0000000000_00000000000000000061_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924332Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000062_00000_0000000001_00000?(+) to d0000000000_00000000000000000062_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924334Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000063_00000_0000000001_00000?(+) to d0000000000_00000000000000000063_00000_0000000001_00000?(+)
2025-11-30T08:31:13.924422Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 64
    Offset: 64
    WriteTimestampMS: 1764491473912
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 10
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.925568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524973:3622] destroyed
2025-11-30T08:31:13.925755Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.926214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524979:3626], now have 1 active actors on pipe
2025-11-30T08:31:13.926369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.926377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.926401Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|aa47c9c9-22480166-58299a23-40b96259_65 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.926409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2279: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-30T08:31:13.926839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 31 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926846Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 32 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926852Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 33 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926857Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 34 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926861Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 35 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926866Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 36 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926872Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 37 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926875Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 38 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926881Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 39 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 40 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926889Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 41 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926893Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 42 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926898Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 43 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926902Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 44 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926907Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 45 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926911Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 46 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926916Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 47 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926920Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 48 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926925Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 49 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926929Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 50 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926933Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 51 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926937Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 52 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926942Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 53 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926946Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 54 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926950Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 55 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926954Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 56 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926959Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 57 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926963Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 58 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926967Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 59 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926974Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 60 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926978Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 61 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926982Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 62 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.926989Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 63 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.927005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.927017Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.927024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.927027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.927033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.927035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.927038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.927046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.927049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.927052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.927064Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.927069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.927079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.927156Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 0 suffix '124' size 262204
2025-11-30T08:31:13.927160Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 32 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927164Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 33 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927170Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 34 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927173Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 35 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927177Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 36 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927180Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 37 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927184Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 38 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927187Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 39 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927190Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 40 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927194Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 41 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927197Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 42 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927201Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 43 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927205Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 44 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927209Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 45 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927212Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 46 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927215Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 47 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927218Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 48 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927222Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 49 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927225Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 50 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927228Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 51 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927232Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 52 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927235Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 53 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927239Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 54 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927242Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 55 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927246Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 56 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927250Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 57 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927253Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 58 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927257Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 59 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927261Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 60 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927264Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 61 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927268Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927271Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 63 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.927470Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.927869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524979:3626] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|aa47c9c9-22480166-58299a23-40b96259_65"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|aa47c9c9-22480166-58299a23-40b96259_65"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 65
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.934082Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.934529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524984:3630], now have 1 active actors on pipe
2025-11-30T08:31:13.934936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.934950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.934972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 65 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.934985Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.934998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.935003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.935011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.935014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.935018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.936266Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 65 partNo 0
2025-11-30T08:31:13.936302Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 65 partNo 0 FormedBlobsCount 0 NewHead: Offset 65 PartNo 0 PackedSize 262214 count 1 nextOffset 66 batches 1
2025-11-30T08:31:13.936310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.936312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.936316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.936418Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 65,1 HeadOffset 65 endOffset 65 curOffset 66 d0000000000_00000000000000000065_00000_0000000001_00000? size 262204 WTime 1764491473936
2025-11-30T08:31:13.936468Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.936480Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 65 partNo 0 count 1 size 262204
2025-11-30T08:31:13.938571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 65 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.938588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.938605Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.938614Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.938625Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 65, partNo: 0, Offset: 65 is stored on disk
2025-11-30T08:31:13.938652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.938655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.938656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.938659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.938661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.938668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.938676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.938787Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 65 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.939165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524984:3630] destroyed
2025-11-30T08:31:13.938850Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 65
    Offset: 65
    WriteTimestampMS: 1764491473934
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.943780Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.944306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524988:3632], now have 1 active actors on pipe
2025-11-30T08:31:13.944477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.944484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.944510Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|d5aed40d-56962db5-7c484300-d96b6c5d_66 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.944518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.944523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.944528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.944530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.944535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.944544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.944547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.944550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.944563Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.944577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.944713Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.945013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524988:3632] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|d5aed40d-56962db5-7c484300-d96b6c5d_66"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|d5aed40d-56962db5-7c484300-d96b6c5d_66"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 66
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.949235Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.949753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524991:3633], now have 1 active actors on pipe
2025-11-30T08:31:13.950199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.950209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.950229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 66 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.950239Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.950251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.950254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.950265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.950267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.950272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.951327Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 66 partNo 0
2025-11-30T08:31:13.951363Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 66 partNo 0 FormedBlobsCount 0 NewHead: Offset 66 PartNo 0 PackedSize 262214 count 1 nextOffset 67 batches 1
2025-11-30T08:31:13.951371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.951373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.951377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.951466Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 66,1 HeadOffset 66 endOffset 66 curOffset 67 d0000000000_00000000000000000066_00000_0000000001_00000? size 262204 WTime 1764491473950
2025-11-30T08:31:13.951507Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.951519Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 66 partNo 0 count 1 size 262204
2025-11-30T08:31:13.961144Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 66 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.961169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.961191Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.961202Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.961214Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 66, partNo: 0, Offset: 66 is stored on disk
2025-11-30T08:31:13.961254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.961257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.961260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.961263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.961269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.961279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.961291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.961365Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 66 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.961467Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 66
    Offset: 66
    WriteTimestampMS: 1764491473949
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 10
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.961974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524991:3633] destroyed
2025-11-30T08:31:13.962505Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.963132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698524998:3638], now have 1 active actors on pipe
2025-11-30T08:31:13.963263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.963272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.963299Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|6be3c1cd-5c4c033f-98af2254-59b6e453_67 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.963308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.963312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.963317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.963319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.963323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.963333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.963338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.963342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.963354Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.963368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.963505Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|6be3c1cd-5c4c033f-98af2254-59b6e453_67"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|6be3c1cd-5c4c033f-98af2254-59b6e453_67"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 67
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.966120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698524998:3638] destroyed
2025-11-30T08:31:13.966195Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.970226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698525001:3639], now have 1 active actors on pipe
2025-11-30T08:31:13.970388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.970401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.970423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 67 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.970442Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.970457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.970465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.970475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.970482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.970486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.971252Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 67 partNo 0
2025-11-30T08:31:13.971291Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 67 partNo 0 FormedBlobsCount 0 NewHead: Offset 67 PartNo 0 PackedSize 262214 count 1 nextOffset 68 batches 1
2025-11-30T08:31:13.971300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.971303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.971306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.971398Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 67,1 HeadOffset 67 endOffset 67 curOffset 68 d0000000000_00000000000000000067_00000_0000000001_00000? size 262204 WTime 1764491473970
2025-11-30T08:31:13.971624Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.971642Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 67 partNo 0 count 1 size 262204
2025-11-30T08:31:13.973982Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 67 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.974006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.974028Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.974042Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.974055Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 67, partNo: 0, Offset: 67 is stored on disk
2025-11-30T08:31:13.974098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.974104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.974109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.974112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.974114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.974122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.974137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.974176Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 67 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.974257Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.974599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698525001:3639] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 67
    Offset: 67
    WriteTimestampMS: 1764491473970
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.975388Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.975747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698525005:3641], now have 1 active actors on pipe
2025-11-30T08:31:13.975905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.975915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.975946Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|451d6d74-79aeb80b-69348e33-8d517fd2_68 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.975958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.975961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.975966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.975968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.975971Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.975978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.975980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.975983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.975994Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.976005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.976082Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|451d6d74-79aeb80b-69348e33-8d517fd2_68"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|451d6d74-79aeb80b-69348e33-8d517fd2_68"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 68
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:13.976966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698525005:3641] destroyed
2025-11-30T08:31:13.978779Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.979201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698525008:3642], now have 1 active actors on pipe
2025-11-30T08:31:13.979643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.979655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.979675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 68 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:13.979689Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:13.979704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.979713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.979721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.979727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.979732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.981329Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 68 partNo 0
2025-11-30T08:31:13.981374Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 68 partNo 0 FormedBlobsCount 0 NewHead: Offset 68 PartNo 0 PackedSize 262214 count 1 nextOffset 69 batches 1
2025-11-30T08:31:13.981385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.981392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.981398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.981512Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 68,1 HeadOffset 68 endOffset 68 curOffset 69 d0000000000_00000000000000000068_00000_0000000001_00000? size 262204 WTime 1764491473981
2025-11-30T08:31:13.981565Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:13.981587Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 68 partNo 0 count 1 size 262204
2025-11-30T08:31:13.986856Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 68 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:13.986882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:13.986904Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:13.986916Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:13.986926Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 68, partNo: 0, Offset: 68 is stored on disk
2025-11-30T08:31:13.986961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.986964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.986967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.986973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:13.986975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.986985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:13.986996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.987138Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 68 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:13.987229Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-30T08:31:13.987660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698525008:3642] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 68
    Offset: 68
    WriteTimestampMS: 1764491473979
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:13.992858Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:13.993568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698525012:3644], now have 1 active actors on pipe
2025-11-30T08:31:13.993591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:13.993597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:13.993632Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|8897924-e90222a8-a235a582-d4d61622_69 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:13.993642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:13.993645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:13.993651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:13.993653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:13.993656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:13.993665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:13.993668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:13.993671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:13.993688Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:13.993705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:13.993861Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:13.994499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698525012:3644] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|8897924-e90222a8-a235a582-d4d61622_69"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|8897924-e90222a8-a235a582-d4d61622_69"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 69
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.000470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433170698525015:3645], now have 1 active actors on pipe
2025-11-30T08:31:14.000580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.000588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.000610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 69 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:14.000622Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:14.000638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.000641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.000652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.000655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.000659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.001505Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 69 partNo 0
2025-11-30T08:31:14.001542Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 69 partNo 0 FormedBlobsCount 0 NewHead: Offset 69 PartNo 0 PackedSize 262214 count 1 nextOffset 70 batches 1
2025-11-30T08:31:14.001553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.001555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.001559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.001656Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 69,1 HeadOffset 69 endOffset 69 curOffset 70 d0000000000_00000000000000000069_00000_0000000001_00000? size 262204 WTime 1764491474001
2025-11-30T08:31:14.001701Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.001715Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 69 partNo 0 count 1 size 262204
2025-11-30T08:31:13.999600Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.009905Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 69 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.009930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.009955Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.009967Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:14.009977Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 69, partNo: 0, Offset: 69 is stored on disk
2025-11-30T08:31:14.010021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.010024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.010026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.010030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.010032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.010042Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.010054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.010235Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 69 partno 0 count 1 parts 0 suffix '63' size 262204
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 69
    Offset: 69
    WriteTimestampMS: 1764491474000
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 8
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:14.011220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433170698525015:3645] destroyed
2025-11-30T08:31:14.010337Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:14.013744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492316:3648], now have 1 active actors on pipe
2025-11-30T08:31:14.013878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.013886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.013926Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|7af870cc-97b5f29b-25e07a33-f1b582ae_70 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:14.013937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.013940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.013945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.013947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.013950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.013960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.013962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.013965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.013982Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:14.014003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.015546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492316:3648] destroyed
2025-11-30T08:31:14.013307Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.014155Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|7af870cc-97b5f29b-25e07a33-f1b582ae_70"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:14.017827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.017838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.017840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.017843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.017845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|7af870cc-97b5f29b-25e07a33-f1b582ae_70"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 70
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.020620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492319:3649], now have 1 active actors on pipe
2025-11-30T08:31:14.020190Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.020993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.021014Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.021037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 70 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:14.021048Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:14.021062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.021067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.021077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.021079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.021083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.022282Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 70 partNo 0
2025-11-30T08:31:14.022315Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 70 partNo 0 FormedBlobsCount 0 NewHead: Offset 70 PartNo 0 PackedSize 262214 count 1 nextOffset 71 batches 1
2025-11-30T08:31:14.022324Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.022326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.022334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.022431Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 70,1 HeadOffset 70 endOffset 70 curOffset 71 d0000000000_00000000000000000070_00000_0000000001_00000? size 262204 WTime 1764491474022
2025-11-30T08:31:14.022476Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.022490Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 70 partNo 0 count 1 size 262204
2025-11-30T08:31:14.027366Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 70 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.027395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.027417Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.027419Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 70 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.027427Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:14.027438Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 70, partNo: 0, Offset: 70 is stored on disk
2025-11-30T08:31:14.027474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.027477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.027479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.027482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.027485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.027493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.027506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.028591Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 70
    Offset: 70
    WriteTimestampMS: 1764491474020
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:14.029193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492319:3649] destroyed
2025-11-30T08:31:14.030152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492323:3651], now have 1 active actors on pipe
2025-11-30T08:31:14.030290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.030299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.030332Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|806095ef-939d647a-257e18da-b4fd98b1_71 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:14.030341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.030344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.030351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.030353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.030357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.030366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.030369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.030371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.030399Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:14.030411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.030718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492323:3651] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|806095ef-939d647a-257e18da-b4fd98b1_71"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|806095ef-939d647a-257e18da-b4fd98b1_71"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 71
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.029646Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.030531Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:14.032685Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.033439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492326:3652], now have 1 active actors on pipe
2025-11-30T08:31:14.033542Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.033549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.033571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 71 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:14.033580Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:14.033594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.033597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.033605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.033609Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.033613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.034397Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 71 partNo 0
2025-11-30T08:31:14.034425Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 71 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 262214 count 1 nextOffset 72 batches 1
2025-11-30T08:31:14.034433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.034436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.034439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.034539Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 71,1 HeadOffset 71 endOffset 71 curOffset 72 d0000000000_00000000000000000071_00000_0000000001_00000? size 262204 WTime 1764491474034
2025-11-30T08:31:14.034582Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.034593Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 71 partNo 0 count 1 size 262204
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 71
    Offset: 71
    WriteTimestampMS: 1764491474033
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 6
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:14.041986Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:14.040928Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 71 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.040968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.040991Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.041000Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:14.041011Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 71, partNo: 0, Offset: 71 is stored on disk
2025-11-30T08:31:14.041049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.041053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.041055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.041058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.041063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.041072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.041084Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.041161Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 71 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.043182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492326:3652] destroyed
2025-11-30T08:31:14.046039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492331:3655], now have 1 active actors on pipe
2025-11-30T08:31:14.046224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.046239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.046271Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|96e1f67d-6e35a7e4-f54069b7-821bbefe_72 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:14.046284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.046292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.046298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.046305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.046309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.046320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.046326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.046330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.046347Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:14.046367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|96e1f67d-6e35a7e4-f54069b7-821bbefe_72"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:14.046937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492331:3655] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|96e1f67d-6e35a7e4-f54069b7-821bbefe_72"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 72
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.045411Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.046479Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:14.049341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492334:3656], now have 1 active actors on pipe
2025-11-30T08:31:14.049881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.049890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.049911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 72 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:14.049921Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:14.049936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.049939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.049948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.049951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.049955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.048907Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.050989Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 72 partNo 0
2025-11-30T08:31:14.051032Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 72 partNo 0 FormedBlobsCount 0 NewHead: Offset 72 PartNo 0 PackedSize 262214 count 1 nextOffset 73 batches 1
2025-11-30T08:31:14.051042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.051045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.051048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.051146Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 72,1 HeadOffset 72 endOffset 72 curOffset 73 d0000000000_00000000000000000072_00000_0000000001_00000? size 262204 WTime 1764491474050
2025-11-30T08:31:14.051195Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.051207Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 72 partNo 0 count 1 size 262204
2025-11-30T08:31:14.061433Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 72 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.061465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.061492Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.061505Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:14.061518Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 72, partNo: 0, Offset: 72 is stored on disk
2025-11-30T08:31:14.061559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.061564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.061567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.061571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.061574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.061585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.061599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.062308Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 72 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.062398Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-30T08:31:14.062783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492334:3656] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 72
    Offset: 72
    WriteTimestampMS: 1764491474049
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 10
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:14.065263Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.069022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492338:3658], now have 1 active actors on pipe
2025-11-30T08:31:14.069249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.069260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.069295Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b4944c25-1d1689d9-679aad47-5e528138_73 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:14.069306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.069312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.069317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.069319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.069325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.069336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.069339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.069342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.069359Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:14.069374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
2025-11-30T08:31:14.069523Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b4944c25-1d1689d9-679aad47-5e528138_73"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:14.069991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492338:3658] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b4944c25-1d1689d9-679aad47-5e528138_73"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 73
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.072777Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.073285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492341:3659], now have 1 active actors on pipe
2025-11-30T08:31:14.073769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.073782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.073807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 73 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:14.073818Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:14.073833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.073838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.073850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.073852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.073856Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.074786Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 73 partNo 0
2025-11-30T08:31:14.074829Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 73 partNo 0 FormedBlobsCount 0 NewHead: Offset 73 PartNo 0 PackedSize 262214 count 1 nextOffset 74 batches 1
2025-11-30T08:31:14.074839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.074842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.074845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.074949Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 73,1 HeadOffset 73 endOffset 73 curOffset 74 d0000000000_00000000000000000073_00000_0000000001_00000? size 262204 WTime 1764491474074
2025-11-30T08:31:14.075000Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.075012Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 73 partNo 0 count 1 size 262204
2025-11-30T08:31:14.086072Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 73 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.086101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.086124Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.086135Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:14.086146Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 73, partNo: 0, Offset: 73 is stored on disk
2025-11-30T08:31:14.086194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.086197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.086200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.086204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.086206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.086216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.086229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.086361Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 73 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.087239Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 73
    Offset: 73
    WriteTimestampMS: 1764491474073
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 11
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:14.088323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492341:3659] destroyed
2025-11-30T08:31:14.088475Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.089367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492345:3661], now have 1 active actors on pipe
2025-11-30T08:31:14.089708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.089732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.089779Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|d2806751-69de9e37-e8623521-fac91744_74 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:14.089793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.089798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.089805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.089808Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.089812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.089826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.089829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.089833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.089855Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:14.089876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.090077Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|d2806751-69de9e37-e8623521-fac91744_74"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:14.090615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492345:3661] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|d2806751-69de9e37-e8623521-fac91744_74"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 74
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.092805Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.093547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492348:3662], now have 1 active actors on pipe
2025-11-30T08:31:14.093855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.093872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.093900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 74 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:14.093914Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:14.093943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.093948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.093960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.093963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.093969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.095067Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 74 partNo 0
2025-11-30T08:31:14.095114Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 74 partNo 0 FormedBlobsCount 0 NewHead: Offset 74 PartNo 0 PackedSize 262214 count 1 nextOffset 75 batches 1
2025-11-30T08:31:14.095128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.095132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.095136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.095276Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 74,1 HeadOffset 74 endOffset 74 curOffset 75 d0000000000_00000000000000000074_00000_0000000001_00000? size 262204 WTime 1764491474094
2025-11-30T08:31:14.095341Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.095353Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 74 partNo 0 count 1 size 262204
2025-11-30T08:31:14.098613Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 74 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.098642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.098665Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 74 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.098669Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.098681Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:14.098693Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 74, partNo: 0, Offset: 74 is stored on disk
2025-11-30T08:31:14.098735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.098744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.098746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.098748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.098751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.098761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.098776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.098874Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:14.099284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492348:3662] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 74
    Offset: 74
    WriteTimestampMS: 1764491474093
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:14.100430Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.100935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492352:3664], now have 1 active actors on pipe
2025-11-30T08:31:14.101045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.101067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.101109Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|d49d8c06-1e2390df-56901907-900eae0f_75 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:14.101127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.101131Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.101136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.101138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.101142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.101154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.101157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.101159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.101177Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:14.101194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.101307Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|d49d8c06-1e2390df-56901907-900eae0f_75"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:14.101715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492352:3664] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|d49d8c06-1e2390df-56901907-900eae0f_75"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 75
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.105719Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.106314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492355:3665], now have 1 active actors on pipe
2025-11-30T08:31:14.106708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.106718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.106746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 75 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:14.106757Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:14.106774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.106779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.106789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.106790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.106794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.107839Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 75 partNo 0
2025-11-30T08:31:14.107885Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 75 partNo 0 FormedBlobsCount 0 NewHead: Offset 75 PartNo 0 PackedSize 262214 count 1 nextOffset 76 batches 1
2025-11-30T08:31:14.107893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.107896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.107900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.108014Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 75,1 HeadOffset 75 endOffset 75 curOffset 76 d0000000000_00000000000000000075_00000_0000000001_00000? size 262204 WTime 1764491474107
2025-11-30T08:31:14.108065Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.108077Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 75 partNo 0 count 1 size 262204
2025-11-30T08:31:14.110516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 75 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.110546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.110570Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 75 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.113126Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 75
    Offset: 75
    WriteTimestampMS: 1764491474106
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:14.115200Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.116086Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|3728d776-fe5211c-54f20703-eed5d28f_76"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:14.112875Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:492: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-11-30T08:31:14.112923Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.112935Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:14.112946Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 75, partNo: 0, Offset: 75 is stored on disk
2025-11-30T08:31:14.112983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.112985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.112988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.112989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.112991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.113000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.113016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.113551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492355:3665] destroyed
2025-11-30T08:31:14.115740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492359:3667], now have 1 active actors on pipe
2025-11-30T08:31:14.115878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.115886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.115937Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|3728d776-fe5211c-54f20703-eed5d28f_76 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:14.115946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.115949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.115955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.115957Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.115961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.115973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.115975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.115981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.115997Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:14.116024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.116406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492359:3667] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|3728d776-fe5211c-54f20703-eed5d28f_76"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 76
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.118539Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.118142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.118156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.118158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.118160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.118162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.119144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492362:3668], now have 1 active actors on pipe
2025-11-30T08:31:14.119615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.119626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.119654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 76 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:14.119665Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:14.119680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.119683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.119695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.119699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.119704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.120890Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 76 partNo 0
2025-11-30T08:31:14.120948Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 76 partNo 0 FormedBlobsCount 0 NewHead: Offset 76 PartNo 0 PackedSize 262214 count 1 nextOffset 77 batches 1
2025-11-30T08:31:14.120961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.120964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.120968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.121082Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 76,1 HeadOffset 76 endOffset 76 curOffset 77 d0000000000_00000000000000000076_00000_0000000001_00000? size 262204 WTime 1764491474119
2025-11-30T08:31:14.121137Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.121149Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 76 partNo 0 count 1 size 262204
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 76
    Offset: 76
    WriteTimestampMS: 1764491474119
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 17
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:14.137008Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 76 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.137043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.137071Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.137075Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 76 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.137082Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:14.137096Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 76, partNo: 0, Offset: 76 is stored on disk
2025-11-30T08:31:14.137138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.137141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.137143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.137145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.137149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.137159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.137173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.137940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492362:3668] destroyed
2025-11-30T08:31:14.137266Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ff4d3119-67639cc4-acf20873-89068c7b_77"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:14.141549Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.143521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492367:3671], now have 1 active actors on pipe
2025-11-30T08:31:14.144090Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:14.143788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.143798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.143847Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ff4d3119-67639cc4-acf20873-89068c7b_77 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:14.143871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.143876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.143884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.143887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.143892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.143905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.143908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.143911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.143932Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:14.143957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.144748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492367:3671] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ff4d3119-67639cc4-acf20873-89068c7b_77"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 77
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.146823Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.147284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492370:3672], now have 1 active actors on pipe
2025-11-30T08:31:14.147681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.147691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.147718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 77 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:14.147732Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:14.147751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.147755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.147769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.147771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.147777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.148889Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 77 partNo 0
2025-11-30T08:31:14.148929Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 77 partNo 0 FormedBlobsCount 0 NewHead: Offset 77 PartNo 0 PackedSize 262214 count 1 nextOffset 78 batches 1
2025-11-30T08:31:14.148939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.148943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.148946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.149058Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 77,1 HeadOffset 77 endOffset 77 curOffset 78 d0000000000_00000000000000000077_00000_0000000001_00000? size 262204 WTime 1764491474148
2025-11-30T08:31:14.149112Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.149132Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 77 partNo 0 count 1 size 262204
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 77
    Offset: 77
    WriteTimestampMS: 1764491474147
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 8
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:14.156971Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 77 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.157014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.157042Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.157041Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 77 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.157053Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:14.157065Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 77, partNo: 0, Offset: 77 is stored on disk
2025-11-30T08:31:14.157111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.157115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.157120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.157122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.157124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.157134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.157146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.157985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492370:3672] destroyed
2025-11-30T08:31:14.159651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492374:3674], now have 1 active actors on pipe
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|d28903c1-fd9a3d6e-c16f9138-28f3ab41_78"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:14.159756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.159770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.159806Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|d28903c1-fd9a3d6e-c16f9138-28f3ab41_78 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:14.159819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.159822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.159829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.159831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.159835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.159847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.159864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.159868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.159889Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:14.159907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.160740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492374:3674] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|d28903c1-fd9a3d6e-c16f9138-28f3ab41_78"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 78
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.157237Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:14.159132Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.159994Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:14.164532Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.165131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492377:3675], now have 1 active actors on pipe
2025-11-30T08:31:14.167622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.167636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.167661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 78 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:14.167674Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:14.167692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.167697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.167710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.167713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.167718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.168578Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 78 partNo 0
2025-11-30T08:31:14.168615Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 78 partNo 0 FormedBlobsCount 0 NewHead: Offset 78 PartNo 0 PackedSize 262214 count 1 nextOffset 79 batches 1
2025-11-30T08:31:14.168623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.168626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.168629Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.168708Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 78,1 HeadOffset 78 endOffset 78 curOffset 79 d0000000000_00000000000000000078_00000_0000000001_00000? size 262204 WTime 1764491474168
2025-11-30T08:31:14.168762Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.168771Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 78 partNo 0 count 1 size 262204
2025-11-30T08:31:14.180102Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 78 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.180136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.180162Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.180177Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:14.180189Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 78, partNo: 0, Offset: 78 is stored on disk
2025-11-30T08:31:14.180231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.180250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.180253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.180255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.180258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.180267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.180279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.180346Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 78 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.180436Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 78
    Offset: 78
    WriteTimestampMS: 1764491474167
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 11
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:14.182857Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.183705Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|adcf4b64-11ac3e34-114a5c7-dffd1c01_79"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|adcf4b64-11ac3e34-114a5c7-dffd1c01_79"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 79
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.181392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492377:3675] destroyed
2025-11-30T08:31:14.183343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492382:3678], now have 1 active actors on pipe
2025-11-30T08:31:14.183455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.183465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.183500Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|adcf4b64-11ac3e34-114a5c7-dffd1c01_79 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:14.183508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.183512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.183517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.183521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.183525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.183533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.183536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.183539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.183553Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:14.183566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.184155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492382:3678] destroyed
2025-11-30T08:31:14.190007Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.196903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492385:3679], now have 1 active actors on pipe
2025-11-30T08:31:14.197082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.197092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.197121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 79 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:14.197140Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:14.197163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.197168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.197181Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.197184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.197189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.198160Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 79 partNo 0
2025-11-30T08:31:14.198202Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 79 partNo 0 FormedBlobsCount 0 NewHead: Offset 79 PartNo 0 PackedSize 262214 count 1 nextOffset 80 batches 1
2025-11-30T08:31:14.198213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.198217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.198221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.198334Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 79,1 HeadOffset 79 endOffset 79 curOffset 80 d0000000000_00000000000000000079_00000_0000000001_00000? size 262204 WTime 1764491474194
2025-11-30T08:31:14.198389Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.198405Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 79 partNo 0 count 1 size 262204
2025-11-30T08:31:14.211700Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 79 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.211743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.211776Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.211796Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:14.211810Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 79, partNo: 0, Offset: 79 is stored on disk
2025-11-30T08:31:14.211871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.211875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.211877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.211879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.211882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.211894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.211911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.212057Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 79 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.212173Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:14.212686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492385:3679] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 79
    Offset: 79
    WriteTimestampMS: 1764491474194
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 13
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:14.217508Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.218088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492392:3683], now have 1 active actors on pipe
2025-11-30T08:31:14.218301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.218312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.218350Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|c3945265-7463248-f3aaa724-775f3567_80 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:14.218363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.218367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.218374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.218376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.218380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.218392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.218395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.218398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.218419Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:14.218441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.218612Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:14.219082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.219089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.219101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.219103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.219106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.219119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492392:3683] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|c3945265-7463248-f3aaa724-775f3567_80"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|c3945265-7463248-f3aaa724-775f3567_80"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 80
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.224096Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.224785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492395:3684], now have 1 active actors on pipe
2025-11-30T08:31:14.225299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.225313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.225338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 80 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:14.225351Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:14.225368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.225373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.225385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.225388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.225394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.226393Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 80 partNo 0
2025-11-30T08:31:14.226436Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 80 partNo 0 FormedBlobsCount 0 NewHead: Offset 80 PartNo 0 PackedSize 262214 count 1 nextOffset 81 batches 1
2025-11-30T08:31:14.226446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.226450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.226460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.226564Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 80,1 HeadOffset 80 endOffset 80 curOffset 81 d0000000000_00000000000000000080_00000_0000000001_00000? size 262204 WTime 1764491474226
2025-11-30T08:31:14.226613Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.226624Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 80 partNo 0 count 1 size 262204
2025-11-30T08:31:14.242105Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:14.241638Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 80 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.241673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.241707Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.241721Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:14.241736Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 80, partNo: 0, Offset: 80 is stored on disk
2025-11-30T08:31:14.241790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.241794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.241796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.241799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.241802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.241814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.241831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.241900Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 80 partno 0 count 1 parts 0 suffix '63' size 262204
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 80
    Offset: 80
    WriteTimestampMS: 1764491474225
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 14
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:14.242610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492395:3684] destroyed
2025-11-30T08:31:14.243523Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.244439Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-30T08:31:14.244091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492399:3686], now have 1 active actors on pipe
2025-11-30T08:31:14.244212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.244222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.244263Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|6c95c44-effacb5e-d9957cfd-52c2606a_81 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:14.244277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|6c95c44-effacb5e-d9957cfd-52c2606a_81"
    Status: Active
    SeqNo: 0
  }
}
2025-11-30T08:31:14.244281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.244287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.244290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:14.244294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.244307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.244310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.244313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.244332Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:14.244349Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|6c95c44-effacb5e-d9957cfd-52c2606a_81"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 81
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.247057Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.247115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492399:3686] destroyed
2025-11-30T08:31:14.247881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492402:3687], now have 1 active actors on pipe
2025-11-30T08:31:14.247979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.247994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.248021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 81 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:14.248034Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:14.248051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.248055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.248067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.248070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.248075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.249338Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 81 partNo 0
2025-11-30T08:31:14.249393Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 81 partNo 0 FormedBlobsCount 0 NewHead: Offset 81 PartNo 0 PackedSize 262214 count 1 nextOffset 82 batches 1
2025-11-30T08:31:14.249410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.249413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.249417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.249500Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 81,1 HeadOffset 81 endOffset 81 curOffset 82 d0000000000_00000000000000000081_00000_0000000001_00000? size 262204 WTime 1764491474248
2025-11-30T08:31:14.249548Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.249561Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 81 partNo 0 count 1 size 262204
2025-11-30T08:31:14.257582Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 81 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.257854Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 81 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.257896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.257932Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.257946Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:14.257961Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 81, partNo: 0, Offset: 81 is stored on disk
2025-11-30T08:31:14.258016Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.258020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.258023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.258027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.258030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.258043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.258062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.258543Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:14.259046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492402:3687] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 81
    Offset: 81
    WriteTimestampMS: 1764491474247
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 8
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:14.260217Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.260890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492406:3689], now have 1 active actors on pipe
2025-11-30T08:31:14.261055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.261066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.261108Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|63eade82-7d075ff7-452fd8a8-3f107502_82 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:14.261120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.261125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.261131Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.261134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.261138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.261150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.261155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.261158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.261482Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:14.261509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.261665Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|63eade82-7d075ff7-452fd8a8-3f107502_82"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|63eade82-7d075ff7-452fd8a8-3f107502_82"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 82
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.263963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492406:3689] destroyed
2025-11-30T08:31:14.264710Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.265808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492409:3690], now have 1 active actors on pipe
2025-11-30T08:31:14.265904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.265917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.265941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 82 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:14.265955Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:14.265973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.265978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.265989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.265991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.265996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.267272Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 82 partNo 0
2025-11-30T08:31:14.267311Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 82 partNo 0 FormedBlobsCount 0 NewHead: Offset 82 PartNo 0 PackedSize 262214 count 1 nextOffset 83 batches 1
2025-11-30T08:31:14.267321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.267326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.267330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.267438Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 82,1 HeadOffset 82 endOffset 82 curOffset 83 d0000000000_00000000000000000082_00000_0000000001_00000? size 262204 WTime 1764491474264
2025-11-30T08:31:14.267490Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.267503Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 82 partNo 0 count 1 size 262204
2025-11-30T08:31:14.276959Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 82 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.276997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.277031Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.277046Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:14.277066Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 82, partNo: 0, Offset: 82 is stored on disk
2025-11-30T08:31:14.277090Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 82 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.277119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.277123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.277126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.277130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.277132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.277145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.277163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.277283Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 82
    Offset: 82
    WriteTimestampMS: 1764491474264
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 12
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:14.278435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492409:3690] destroyed
2025-11-30T08:31:14.278644Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.279147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492413:3692], now have 1 active actors on pipe
2025-11-30T08:31:14.279311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.279322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.279361Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|94fa6113-45277ec6-8c4a280c-7cd6caeb_83 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:14.279384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.279389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.279395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.279398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.279402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.279414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.279416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.279420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.279438Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:14.279453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.279606Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|94fa6113-45277ec6-8c4a280c-7cd6caeb_83"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
2025-11-30T08:31:14.280783Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492413:3692] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|94fa6113-45277ec6-8c4a280c-7cd6caeb_83"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 83
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.282960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492416:3693], now have 1 active actors on pipe
2025-11-30T08:31:14.282498Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.283489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.283501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.283524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 83 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:14.283536Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:14.283550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.283554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.283564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.283567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.283572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.284704Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 83 partNo 0
2025-11-30T08:31:14.284740Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 83 partNo 0 FormedBlobsCount 0 NewHead: Offset 83 PartNo 0 PackedSize 262214 count 1 nextOffset 84 batches 1
2025-11-30T08:31:14.284748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.284751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.284755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.284869Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 83,1 HeadOffset 83 endOffset 83 curOffset 84 d0000000000_00000000000000000083_00000_0000000001_00000? size 262204 WTime 1764491474283
2025-11-30T08:31:14.284918Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.284931Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 83 partNo 0 count 1 size 262204
2025-11-30T08:31:14.290795Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 83 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.290821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.290843Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.290853Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:14.290865Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 83, partNo: 0, Offset: 83 is stored on disk
2025-11-30T08:31:14.290913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.290916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.290918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.290921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.290924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.290933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.290944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.291082Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 83 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.291194Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 83
    Offset: 83
    WriteTimestampMS: 1764491474283
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 7
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-30T08:31:14.291779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492416:3693] destroyed
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:14.292534Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.293087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492421:3696], now have 1 active actors on pipe
2025-11-30T08:31:14.293237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.293246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.293276Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|45c4e10d-fd288fbe-2801ce60-79956e60_84 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:14.293285Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.293288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.293292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.293294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.293298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.293308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.293310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.293313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.293326Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:14.293339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.293494Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:14.293795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492421:3696] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|45c4e10d-fd288fbe-2801ce60-79956e60_84"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|45c4e10d-fd288fbe-2801ce60-79956e60_84"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 84
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.296193Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.296808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492424:3697], now have 1 active actors on pipe
2025-11-30T08:31:14.297324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.297334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.297356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 84 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:14.297365Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:14.297378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.297381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.297391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.297396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.297402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.298470Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 84 partNo 0
2025-11-30T08:31:14.298499Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 84 partNo 0 FormedBlobsCount 0 NewHead: Offset 84 PartNo 0 PackedSize 262214 count 1 nextOffset 85 batches 1
2025-11-30T08:31:14.298508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.298510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.298513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.298610Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 84,1 HeadOffset 84 endOffset 84 curOffset 85 d0000000000_00000000000000000084_00000_0000000001_00000? size 262204 WTime 1764491474297
2025-11-30T08:31:14.298653Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.298665Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 84 partNo 0 count 1 size 262204
2025-11-30T08:31:14.302663Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 84 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.302680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.302698Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.302705Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:14.302716Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 84, partNo: 0, Offset: 84 is stored on disk
2025-11-30T08:31:14.302752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.302754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.302757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.302760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.302762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.302769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.302782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.302905Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 84 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.303015Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:14.303418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492424:3697] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 84
    Offset: 84
    WriteTimestampMS: 1764491474297
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:14.304292Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.304855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492428:3699], now have 1 active actors on pipe
2025-11-30T08:31:14.305295Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|87103db5-24ab9782-6f46a79-47d8cf08_85"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|87103db5-24ab9782-6f46a79-47d8cf08_85"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 85
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.308260Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.305007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.305024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.305053Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|87103db5-24ab9782-6f46a79-47d8cf08_85 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:14.305060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.305064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.305067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.305070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.305073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.305081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.305083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.305086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.305099Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:14.305110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.305782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492428:3699] destroyed
2025-11-30T08:31:14.308736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492431:3700], now have 1 active actors on pipe
2025-11-30T08:31:14.309228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.309247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.309264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 85 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:14.309273Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:14.309283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.309286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.309294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.309299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.309303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.310197Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 85 partNo 0
2025-11-30T08:31:14.310228Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 85 partNo 0 FormedBlobsCount 0 NewHead: Offset 85 PartNo 0 PackedSize 262214 count 1 nextOffset 86 batches 1
2025-11-30T08:31:14.310236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.310239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.310242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.310336Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 85,1 HeadOffset 85 endOffset 85 curOffset 86 d0000000000_00000000000000000085_00000_0000000001_00000? size 262204 WTime 1764491474306
2025-11-30T08:31:14.310381Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.310392Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 85 partNo 0 count 1 size 262204
2025-11-30T08:31:14.312874Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 85 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.312889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.312914Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.312921Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:14.312930Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 85, partNo: 0, Offset: 85 is stored on disk
2025-11-30T08:31:14.312960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.312974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.312976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.312978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.312980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.312987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.312998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.313154Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 85
    Offset: 85
    WriteTimestampMS: 1764491474306
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:14.314253Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.315017Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|27bf7dfa-3609bccd-d90b63ca-f0b19688_86"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|27bf7dfa-3609bccd-d90b63ca-f0b19688_86"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 86
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.313060Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 85 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.314305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492431:3700] destroyed
2025-11-30T08:31:14.314819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492435:3702], now have 1 active actors on pipe
2025-11-30T08:31:14.314826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.314831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.314856Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|27bf7dfa-3609bccd-d90b63ca-f0b19688_86 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:14.314862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.314867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.314871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.314876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.314879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.314887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.314889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.314891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.314900Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:14.314910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.316649Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492435:3702] destroyed
2025-11-30T08:31:14.318205Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.319697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.319712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.319715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.319717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.319720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.319735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492438:3703], now have 1 active actors on pipe
2025-11-30T08:31:14.319843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.319865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.319883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 86 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:14.319891Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:14.319901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.319904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.319911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.319913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.319917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.324244Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 86
    Offset: 86
    WriteTimestampMS: 1764491474319
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-30T08:31:14.321248Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 86 partNo 0
2025-11-30T08:31:14.321295Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 86 partNo 0 FormedBlobsCount 0 NewHead: Offset 86 PartNo 0 PackedSize 262214 count 1 nextOffset 87 batches 1
2025-11-30T08:31:14.321304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.321307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.321311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:14.325406Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.326193Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|71891e57-1e440e83-a54394e4-94cf1e14_87"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|71891e57-1e440e83-a54394e4-94cf1e14_87"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 87
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.327806Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.321409Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 86,1 HeadOffset 86 endOffset 86 curOffset 87 d0000000000_00000000000000000086_00000_0000000001_00000? size 262204 WTime 1764491474320
2025-11-30T08:31:14.321469Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.321480Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 86 partNo 0 count 1 size 262204
2025-11-30T08:31:14.323995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 86 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.324009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.324028Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.324038Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:14.324047Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 86, partNo: 0, Offset: 86 is stored on disk
2025-11-30T08:31:14.324078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.324080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.324083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.324085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.324086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.324092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.324101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.324154Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 86 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.325144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492438:3703] destroyed
2025-11-30T08:31:14.326059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492442:3705], now have 1 active actors on pipe
2025-11-30T08:31:14.326066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.326069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.326087Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|71891e57-1e440e83-a54394e4-94cf1e14_87 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:14.326092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.326094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.326097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.326099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.326102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.326107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.326109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.326111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.326119Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:14.326127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.327283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492442:3705] destroyed
2025-11-30T08:31:14.328493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492445:3706], now have 1 active actors on pipe
2025-11-30T08:31:14.328589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.328593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.328611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 87 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:14.328619Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:14.328629Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.328631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.328642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.328645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.328649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.329658Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 87 partNo 0
2025-11-30T08:31:14.329698Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 87 partNo 0 FormedBlobsCount 0 NewHead: Offset 87 PartNo 0 PackedSize 262214 count 1 nextOffset 88 batches 1
2025-11-30T08:31:14.329705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.329707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.329709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.329798Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 87,1 HeadOffset 87 endOffset 87 curOffset 88 d0000000000_00000000000000000087_00000_0000000001_00000? size 262204 WTime 1764491474328
2025-11-30T08:31:14.329840Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.329851Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 87 partNo 0 count 1 size 262204
2025-11-30T08:31:14.332041Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 87 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.332063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.332105Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 87 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.333304Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:492: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-11-30T08:31:14.333335Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.333344Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:14.333358Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 87, partNo: 0, Offset: 87 is stored on disk
2025-11-30T08:31:14.333392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.333399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.333401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.333428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.333430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.333440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.333452Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.333560Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 87
    Offset: 87
    WriteTimestampMS: 1764491474328
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:14.333881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492445:3706] destroyed
2025-11-30T08:31:14.334525Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.334920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492449:3708], now have 1 active actors on pipe
2025-11-30T08:31:14.335589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.335603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.335635Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|71a8d1b-769b0162-5c3cdcea-cdc251ac_88 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:14.335646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.335652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.335656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.335657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.335660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.335668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.335670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.335673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.335683Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:14.335694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.335750Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:14.335959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492449:3708] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|71a8d1b-769b0162-5c3cdcea-cdc251ac_88"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|71a8d1b-769b0162-5c3cdcea-cdc251ac_88"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 88
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.338051Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.338498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492452:3709], now have 1 active actors on pipe
2025-11-30T08:31:14.339015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.339029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.339058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 88 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:14.339071Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:14.339082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.339088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.339095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.339100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.339103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.340089Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 88 partNo 0
2025-11-30T08:31:14.340125Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 88 partNo 0 FormedBlobsCount 0 NewHead: Offset 88 PartNo 0 PackedSize 262214 count 1 nextOffset 89 batches 1
2025-11-30T08:31:14.340136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.340138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.340141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.340225Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 88,1 HeadOffset 88 endOffset 88 curOffset 89 d0000000000_00000000000000000088_00000_0000000001_00000? size 262204 WTime 1764491474338
2025-11-30T08:31:14.340272Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.340288Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 88 partNo 0 count 1 size 262204
2025-11-30T08:31:14.343097Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 88 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.343120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.343144Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 88 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.343148Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.343160Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:14.343169Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 88, partNo: 0, Offset: 88 is stored on disk
2025-11-30T08:31:14.343200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.343202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.343204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.343206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.343209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.343215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.343226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.343303Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 88
    Offset: 88
    WriteTimestampMS: 1764491474338
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:14.343834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492452:3709] destroyed
2025-11-30T08:31:14.345155Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.345599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492457:3712], now have 1 active actors on pipe
2025-11-30T08:31:14.345703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.345715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.345759Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|fb95f213-6a1b91cd-382577b4-e1935ffe_89 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:14.345770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.345773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.345778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.345780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.345783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.345792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.345797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.345800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.345811Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:14.345821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.345871Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-30T08:31:14.346109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492457:3712] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|fb95f213-6a1b91cd-382577b4-e1935ffe_89"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|fb95f213-6a1b91cd-382577b4-e1935ffe_89"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 89
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.349738Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.350175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492460:3713], now have 1 active actors on pipe
2025-11-30T08:31:14.351426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.351438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.351458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 89 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:14.351469Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:14.351482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.351486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.351495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.351499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.351503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.352594Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 89 partNo 0
2025-11-30T08:31:14.352631Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 89 partNo 0 FormedBlobsCount 0 NewHead: Offset 89 PartNo 0 PackedSize 262214 count 1 nextOffset 90 batches 1
2025-11-30T08:31:14.352639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.352642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.352646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.352738Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 89,1 HeadOffset 89 endOffset 89 curOffset 90 d0000000000_00000000000000000089_00000_0000000001_00000? size 262204 WTime 1764491474351
2025-11-30T08:31:14.352783Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.359845Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 89 partNo 0 count 1 size 262204
2025-11-30T08:31:14.372719Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 89 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.372765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.372891Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.372944Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:14.372995Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 89, partNo: 0, Offset: 89 is stored on disk
2025-11-30T08:31:14.373095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.373113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.373130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.373151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.373162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.373191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.373233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.373393Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 89 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.373508Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:14.374135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492460:3713] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 89
    Offset: 89
    WriteTimestampMS: 1764491474349
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 21
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:14.375725Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.376492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492464:3715], now have 1 active actors on pipe
2025-11-30T08:31:14.376554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.376582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.376627Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|1fc010d7-adc7f090-c1099f11-2b178a1_90 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:14.376650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.376661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.376669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.376671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.376682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.376697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.376700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.376702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.376714Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:14.376729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.376831Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-30T08:31:14.377161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492464:3715] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|1fc010d7-adc7f090-c1099f11-2b178a1_90"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|1fc010d7-adc7f090-c1099f11-2b178a1_90"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 90
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.381681Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.382175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492467:3716], now have 1 active actors on pipe
2025-11-30T08:31:14.382951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.382981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.383016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 90 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:14.383028Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:14.383042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.383046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.383055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.383057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.383061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.384370Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 90 partNo 0
2025-11-30T08:31:14.384409Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 90 partNo 0 FormedBlobsCount 0 NewHead: Offset 90 PartNo 0 PackedSize 262214 count 1 nextOffset 91 batches 1
2025-11-30T08:31:14.384417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.384420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.384423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.384504Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 90,1 HeadOffset 90 endOffset 90 curOffset 91 d0000000000_00000000000000000090_00000_0000000001_00000? size 262204 WTime 1764491474384
2025-11-30T08:31:14.384577Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.384587Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 90 partNo 0 count 1 size 262204
2025-11-30T08:31:14.396905Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 90 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.396936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.396959Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 90 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.396982Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.396994Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:14.397010Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 90, partNo: 0, Offset: 90 is stored on disk
2025-11-30T08:31:14.397052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.397062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.397064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.397066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.397068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.397078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.397090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 90
    Offset: 90
    WriteTimestampMS: 1764491474382
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 12
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-30T08:31:14.398681Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:14.399737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492467:3716] destroyed
2025-11-30T08:31:14.400220Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.400641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492472:3719], now have 1 active actors on pipe
2025-11-30T08:31:14.400785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.400816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.400880Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|628de9be-b75c7787-14067500-4496c71e_91 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:14.400888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.400893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.400898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.400900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.400904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.400912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.400919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.400922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.400936Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:14.400947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.401014Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|628de9be-b75c7787-14067500-4496c71e_91"
    Status: Active
    SeqNo: 0
  }
}
2025-11-30T08:31:14.401249Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492472:3719] destroyed
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|628de9be-b75c7787-14067500-4496c71e_91"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 91
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.404122Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.404574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492475:3720], now have 1 active actors on pipe
2025-11-30T08:31:14.405039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.405052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.405074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 91 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:14.405084Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:14.405097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.405100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.405128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.405130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.405134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.406713Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 91 partNo 0
2025-11-30T08:31:14.406743Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 91 partNo 0 FormedBlobsCount 0 NewHead: Offset 91 PartNo 0 PackedSize 262214 count 1 nextOffset 92 batches 1
2025-11-30T08:31:14.406751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.406754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.406910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.407009Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 91,1 HeadOffset 91 endOffset 91 curOffset 92 d0000000000_00000000000000000091_00000_0000000001_00000? size 262204 WTime 1764491474406
2025-11-30T08:31:14.407054Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.407066Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 91 partNo 0 count 1 size 262204
2025-11-30T08:31:14.409700Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 91 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.409723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.409742Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.409742Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 91 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.409753Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:14.409761Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 91, partNo: 0, Offset: 91 is stored on disk
2025-11-30T08:31:14.409786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.409792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.409794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.409798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.409800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.409807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.409823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.409968Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 91
    Offset: 91
    WriteTimestampMS: 1764491474405
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 2
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:14.410355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492475:3720] destroyed
2025-11-30T08:31:14.410985Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.411402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492479:3722], now have 1 active actors on pipe
2025-11-30T08:31:14.411506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.411514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.411538Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|adda3eee-2edaf5a8-26a35792-d2724836_92 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:14.411549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.411552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.411556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.411558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.411561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.411568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.411570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.411572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.411597Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:14.411609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.411715Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:14.411915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492479:3722] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|adda3eee-2edaf5a8-26a35792-d2724836_92"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|adda3eee-2edaf5a8-26a35792-d2724836_92"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 92
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.414247Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.414709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492482:3723], now have 1 active actors on pipe
2025-11-30T08:31:14.415175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.415185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.415231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 92 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:14.415240Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:14.415251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.415254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.415265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.415266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.415270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.416382Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 92 partNo 0
2025-11-30T08:31:14.416417Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 92 partNo 0 FormedBlobsCount 0 NewHead: Offset 92 PartNo 0 PackedSize 262214 count 1 nextOffset 93 batches 1
2025-11-30T08:31:14.416425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.416427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.416430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.416519Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 92,1 HeadOffset 92 endOffset 92 curOffset 93 d0000000000_00000000000000000092_00000_0000000001_00000? size 262204 WTime 1764491474416
2025-11-30T08:31:14.416560Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.416571Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 92 partNo 0 count 1 size 262204
2025-11-30T08:31:14.419097Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 92 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.419126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.419146Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.419160Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:14.419165Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 92 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.419169Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 92, partNo: 0, Offset: 92 is stored on disk
2025-11-30T08:31:14.419206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.419208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.419210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.419212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.419214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.419225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.419238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.419383Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-30T08:31:14.419778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492482:3723] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 92
    Offset: 92
    WriteTimestampMS: 1764491474414
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:14.419990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.419998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.420000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.420051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.420059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.420565Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.421503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492486:3725], now have 1 active actors on pipe
2025-11-30T08:31:14.421637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.421648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.421680Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|404a42b1-3c11d0ce-45937bff-70ce032c_93 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:14.421690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.421693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.421697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.421700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.421703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.421711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.421714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.421716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.421727Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:14.421737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.421829Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|404a42b1-3c11d0ce-45937bff-70ce032c_93"
    Status: Active
    SeqNo: 0
  }
}
2025-11-30T08:31:14.422076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492486:3725] destroyed
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|404a42b1-3c11d0ce-45937bff-70ce032c_93"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 93
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.424874Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.425298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492489:3726], now have 1 active actors on pipe
2025-11-30T08:31:14.425695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.425714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.425732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 93 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:14.425746Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:14.425760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.425770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.425779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.425785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.425789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.426596Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 93 partNo 0
2025-11-30T08:31:14.426640Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 93 partNo 0 FormedBlobsCount 0 NewHead: Offset 93 PartNo 0 PackedSize 262214 count 1 nextOffset 94 batches 1
2025-11-30T08:31:14.426652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.426657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.426662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.426730Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 93,1 HeadOffset 93 endOffset 93 curOffset 94 d0000000000_00000000000000000093_00000_0000000001_00000? size 262204 WTime 1764491474426
2025-11-30T08:31:14.426768Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.426780Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 93 partNo 0 count 1 size 262204
2025-11-30T08:31:14.428911Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 93 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.428929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.428983Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 93 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.429025Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.429049Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:14.429062Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 93, partNo: 0, Offset: 93 is stored on disk
2025-11-30T08:31:14.429095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.429101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.429103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.429106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.429108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.429115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.429134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.429193Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 93
    Offset: 93
    WriteTimestampMS: 1764491474425
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2025-11-30T08:31:14.429446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492489:3726] destroyed
30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:14.430043Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.430484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492493:3728], now have 1 active actors on pipe
2025-11-30T08:31:14.430579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.430589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.430614Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9ecaabcc-fe3027de-b5469a99-c12db250_94 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:14.430624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.430627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.430632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.430634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.430637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.430644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.430650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.430653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.430663Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:14.430671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.430747Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-30T08:31:14.430932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492493:3728] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9ecaabcc-fe3027de-b5469a99-c12db250_94"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9ecaabcc-fe3027de-b5469a99-c12db250_94"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 94
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.432983Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.433673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492496:3729], now have 1 active actors on pipe
2025-11-30T08:31:14.433737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.433741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.433754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 94 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:14.433760Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:14.433768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.433771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.433777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.433778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.433781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.434354Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 94 partNo 0
2025-11-30T08:31:14.434375Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 94 partNo 0 FormedBlobsCount 0 NewHead: Offset 94 PartNo 0 PackedSize 262214 count 1 nextOffset 95 batches 1
2025-11-30T08:31:14.434380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.434382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.434383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.434443Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 94,1 HeadOffset 94 endOffset 94 curOffset 95 d0000000000_00000000000000000094_00000_0000000001_00000? size 262204 WTime 1764491474434
2025-11-30T08:31:14.434474Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.434482Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 94 partNo 0 count 1 size 262204
2025-11-30T08:31:14.436510Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 94 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.436525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.436541Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.436547Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:14.436555Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 94, partNo: 0, Offset: 94 is stored on disk
2025-11-30T08:31:14.436579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.436580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.436582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.436629Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.436632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.436638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.436646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.436761Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 94 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.436879Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:14.437238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492496:3729] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 94
    Offset: 94
    WriteTimestampMS: 1764491474433
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-30T08:31:14.441492Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.442078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492500:3731], now have 1 active actors on pipe
2025-11-30T08:31:14.442097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.442101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.442131Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f9d23f9-dbaa2be1-35112f2c-8198c160_95 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-30T08:31:14.442140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.442142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.442146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.442148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.442151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.442159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.442161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.442164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.442240Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:14.442176Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:37: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-30T08:31:14.442188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f9d23f9-dbaa2be1-35112f2c-8198c160_95"
    Status: Active
    SeqNo: 0
  }
}
2025-11-30T08:31:14.442526Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492500:3731] destroyed
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f9d23f9-dbaa2be1-35112f2c-8198c160_95"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 95
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.445996Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.449470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492503:3732], now have 1 active actors on pipe
2025-11-30T08:31:14.449674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.449695Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.449725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 95 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-30T08:31:14.449751Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:636: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-30T08:31:14.449791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.449801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2397: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-30T08:31:14.449817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-30T08:31:14.449826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.449832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2461: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-30T08:31:14.450983Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1342: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 95 partNo 0
2025-11-30T08:31:14.451047Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1446: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 95 partNo 0 FormedBlobsCount 0 NewHead: Offset 95 PartNo 0 PackedSize 262214 count 1 nextOffset 96 batches 1
2025-11-30T08:31:14.451062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-30T08:31:14.451066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2316: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-30T08:31:14.451070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.451186Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1698: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 95,1 HeadOffset 95 endOffset 95 curOffset 96 d0000000000_00000000000000000095_00000_0000000001_00000? size 262204 WTime 1764491474450
2025-11-30T08:31:14.451247Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.451256Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 95 partNo 0 count 1 size 262204
2025-11-30T08:31:14.461561Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 95 count 1 size 262204 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.461591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.461613Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.461623Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:60: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-30T08:31:14.461624Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 95 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.461634Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:365: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 95, partNo: 0, Offset: 95 is stored on disk
2025-11-30T08:31:14.461670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.461673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.461675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.461677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.461679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.461689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000064_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000065_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461725Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000066_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461727Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000067_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000068_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000069_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000070_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461739Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000071_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000072_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461746Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000073_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000074_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000075_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000076_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000077_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000078_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000079_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000080_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000081_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000082_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000083_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000084_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000085_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000086_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000087_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000088_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000089_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000090_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000091_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000092_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000093_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000094_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000095_00000_0000000001_00000? (262204)
2025-11-30T08:31:14.461876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 32 keys were taken away. Let's read 8390528 bytes
2025-11-30T08:31:14.461889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.461904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 32 blobs
2025-11-30T08:31:14.461906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000064_00000_0000000001_00000?
2025-11-30T08:31:14.461918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000065_00000_0000000001_00000?
2025-11-30T08:31:14.461937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000066_00000_0000000001_00000?
2025-11-30T08:31:14.461953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000067_00000_0000000001_00000?
2025-11-30T08:31:14.461959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000068_00000_0000000001_00000?
2025-11-30T08:31:14.461977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000069_00000_0000000001_00000?
2025-11-30T08:31:14.461978Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:14.461984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000070_00000_0000000001_00000?
2025-11-30T08:31:14.461996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000071_00000_0000000001_00000?
2025-11-30T08:31:14.462036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000072_00000_0000000001_00000?
2025-11-30T08:31:14.462067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000073_00000_0000000001_00000?
2025-11-30T08:31:14.462087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000074_00000_0000000001_00000?
2025-11-30T08:31:14.462098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000075_00000_0000000001_00000?
2025-11-30T08:31:14.462109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000076_00000_0000000001_00000?
2025-11-30T08:31:14.462120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000077_00000_0000000001_00000?
2025-11-30T08:31:14.462127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000078_00000_0000000001_00000?
2025-11-30T08:31:14.462135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000079_00000_0000000001_00000?
2025-11-30T08:31:14.462143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000080_00000_0000000001_00000?
2025-11-30T08:31:14.462189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000081_00000_0000000001_00000?
2025-11-30T08:31:14.462200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000082_00000_0000000001_00000?
2025-11-30T08:31:14.462211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000083_00000_0000000001_00000?
2025-11-30T08:31:14.462228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000084_00000_0000000001_00000?
2025-11-30T08:31:14.462242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000085_00000_0000000001_00000?
2025-11-30T08:31:14.462265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000086_00000_0000000001_00000?
2025-11-30T08:31:14.462274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000087_00000_0000000001_00000?
2025-11-30T08:31:14.462284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000088_00000_0000000001_00000?
2025-11-30T08:31:14.462290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000089_00000_0000000001_00000?
2025-11-30T08:31:14.462300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000090_00000_0000000001_00000?
2025-11-30T08:31:14.462307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000091_00000_0000000001_00000?
2025-11-30T08:31:14.462328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000092_00000_0000000001_00000?
2025-11-30T08:31:14.462338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000093_00000_0000000001_00000?
2025-11-30T08:31:14.462345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000094_00000_0000000001_00000?
2025-11-30T08:31:14.462357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000095_00000_0000000001_00000?
2025-11-30T08:31:14.462368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 32 blobs for compaction
2025-11-30T08:31:14.462381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492503:3732] destroyed
CallPersQueueGRPC response:
2025-11-30T08:31:14.462393Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 64 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:14.462399Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 65 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:14.462423Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 66 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:14.462429Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 67 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:14.462432Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 68 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:14.462436Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 69 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:14.000000Z
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 95
    Offset: 95
    WriteTimestampMS: 1764491474449
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 10
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-30T08:31:14.462450Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 70 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:14.000000Z
2025-11-30T08:31:14.462457Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 71 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:14.000000Z
2025-11-30T08:31:14.462461Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 72 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:14.000000Z
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdRead {
    ClientId: "user1"
    Offset: 0
    Count: 48
    ReadTimestampMs: 0
  }
}
2025-11-30T08:31:14.462473Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 73 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:14.000000Z
2025-11-30T08:31:14.462481Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 74 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:14.000000Z
2025-11-30T08:31:14.462485Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 75 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:14.000000Z
2025-11-30T08:31:14.462487Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 76 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:14.000000Z
2025-11-30T08:31:14.462492Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 77 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:14.000000Z
2025-11-30T08:31:14.462495Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 78 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:14.000000Z
2025-11-30T08:31:14.462497Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 79 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:14.000000Z
2025-11-30T08:31:14.462510Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 80 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:14.000000Z
2025-11-30T08:31:14.462513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 81 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:14.000000Z
2025-11-30T08:31:14.462516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 82 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:14.000000Z
2025-11-30T08:31:14.462518Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 83 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:14.000000Z
2025-11-30T08:31:14.462520Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 84 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:14.000000Z
2025-11-30T08:31:14.462523Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 85 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:14.000000Z
2025-11-30T08:31:14.462534Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 86 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:14.000000Z
2025-11-30T08:31:14.462537Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 87 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:14.000000Z
2025-11-30T08:31:14.462539Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 88 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:14.000000Z
2025-11-30T08:31:14.462542Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 89 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:14.000000Z
2025-11-30T08:31:14.462545Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 90 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:14.000000Z
2025-11-30T08:31:14.462547Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 91 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:14.000000Z
2025-11-30T08:31:14.462550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 92 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:14.000000Z
2025-11-30T08:31:14.462551Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 93 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:14.000000Z
2025-11-30T08:31:14.462554Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 94 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:14.000000Z
2025-11-30T08:31:14.462556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 95 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-30T08:31:14.000000Z
2025-11-30T08:31:14.462571Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 32 blobs are from cache.
2025-11-30T08:31:14.462595Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 64 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462599Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 65 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462626Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 66 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462630Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 67 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462634Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 68 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462647Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 69 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462650Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 70 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462654Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 71 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462684Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 72 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462688Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 73 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462691Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 74 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462695Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 75 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462702Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 76 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462707Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 77 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462711Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 78 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462715Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 79 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462718Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 80 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462721Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 81 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462724Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 82 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462730Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 83 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462746Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 84 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462749Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 85 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462753Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 86 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462756Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 87 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462759Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 88 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-30T08:31:14.462765Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 89 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462769Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 90 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462772Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 91 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462774Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 92 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462778Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 93 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462781Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 94 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/32] d0000000000_00000000000000000064_00000_0000000001_00000?
2025-11-30T08:31:14.462785Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 95 partno 0 count 1 parts 0 suffix '63'
2025-11-30T08:31:14.462786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.462788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000064_00000_0000000001_00000?, parameters.CurOffset=64
2025-11-30T08:31:14.462969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 64.0/1
2025-11-30T08:31:14.462987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 64 partNo 0
2025-11-30T08:31:14.463009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 64 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 262214 count 1 nextOffset 65 batches 1
2025-11-30T08:31:14.463017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 64:0 appended
2025-11-30T08:31:14.463024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/32] d0000000000_00000000000000000065_00000_0000000001_00000?
2025-11-30T08:31:14.463026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.463029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000065_00000_0000000001_00000?, parameters.CurOffset=65
2025-11-30T08:31:14.463254Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.463842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 65.0/1
2025-11-30T08:31:14.463867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 65 partNo 0
2025-11-30T08:31:14.464196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 65 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 524387 count 2 nextOffset 66 batches 1
2025-11-30T08:31:14.464207Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 65:0 appended
2025-11-30T08:31:14.464213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/32] d0000000000_00000000000000000066_00000_0000000001_00000?
2025-11-30T08:31:14.464215Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.464217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000066_00000_0000000001_00000?, parameters.CurOffset=66
2025-11-30T08:31:14.464361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 66.0/1
2025-11-30T08:31:14.464371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 66 partNo 0
2025-11-30T08:31:14.464380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 66 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 786601 count 3 nextOffset 67 batches 2
2025-11-30T08:31:14.464382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 66:0 appended
2025-11-30T08:31:14.464384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/32] d0000000000_00000000000000000067_00000_0000000001_00000?
2025-11-30T08:31:14.464385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.464388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000067_00000_0000000001_00000?, parameters.CurOffset=67
2025-11-30T08:31:14.464512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 67.0/1
2025-11-30T08:31:14.464521Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 67 partNo 0
2025-11-30T08:31:14.465021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 67 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 1048774 count 4 nextOffset 68 batches 2
2025-11-30T08:31:14.465033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 67:0 appended
2025-11-30T08:31:14.465037Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/32] d0000000000_00000000000000000068_00000_0000000001_00000?
2025-11-30T08:31:14.465038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.465040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000068_00000_0000000001_00000?, parameters.CurOffset=68
2025-11-30T08:31:14.465160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 68.0/1
2025-11-30T08:31:14.465173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 68 partNo 0
2025-11-30T08:31:14.465182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 68 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 1310988 count 5 nextOffset 69 batches 3
2025-11-30T08:31:14.465183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 68:0 appended
2025-11-30T08:31:14.465185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/32] d0000000000_00000000000000000069_00000_0000000001_00000?
2025-11-30T08:31:14.465187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.465188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000069_00000_0000000001_00000?, parameters.CurOffset=69
2025-11-30T08:31:14.465303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 69.0/1
2025-11-30T08:31:14.465312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 69 partNo 0
2025-11-30T08:31:14.466784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 69 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 1573161 count 6 nextOffset 70 batches 3
2025-11-30T08:31:14.466796Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 69:0 appended
2025-11-30T08:31:14.466799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/32] d0000000000_00000000000000000070_00000_0000000001_00000?
2025-11-30T08:31:14.466800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.466802Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000070_00000_0000000001_00000?, parameters.CurOffset=70
2025-11-30T08:31:14.466939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 70.0/1
2025-11-30T08:31:14.466949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 70 partNo 0
2025-11-30T08:31:14.466974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 70 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 1835375 count 7 nextOffset 71 batches 4
2025-11-30T08:31:14.466982Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 70:0 appended
2025-11-30T08:31:14.466986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/32] d0000000000_00000000000000000071_00000_0000000001_00000?
2025-11-30T08:31:14.466988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.466993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000071_00000_0000000001_00000?, parameters.CurOffset=71
2025-11-30T08:31:14.467135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 71.0/1
2025-11-30T08:31:14.467149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 71 partNo 0
2025-11-30T08:31:14.468237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 71 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 2097548 count 8 nextOffset 72 batches 4
2025-11-30T08:31:14.468247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 71:0 appended
2025-11-30T08:31:14.468251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[8/32] d0000000000_00000000000000000072_00000_0000000001_00000?
2025-11-30T08:31:14.468252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.468254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000072_00000_0000000001_00000?, parameters.CurOffset=72
2025-11-30T08:31:14.468376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 72.0/1
2025-11-30T08:31:14.468388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 72 partNo 0
2025-11-30T08:31:14.468399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 72 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 2359762 count 9 nextOffset 73 batches 5
2025-11-30T08:31:14.468402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 72:0 appended
2025-11-30T08:31:14.468406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[9/32] d0000000000_00000000000000000073_00000_0000000001_00000?
2025-11-30T08:31:14.468407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.468408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000073_00000_0000000001_00000?, parameters.CurOffset=73
2025-11-30T08:31:14.468547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 73.0/1
2025-11-30T08:31:14.468559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 73 partNo 0
2025-11-30T08:31:14.470889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 73 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 2621935 count 10 nextOffset 74 batches 5
2025-11-30T08:31:14.470904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 73:0 appended
2025-11-30T08:31:14.470909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[10/32] d0000000000_00000000000000000074_00000_0000000001_00000?
2025-11-30T08:31:14.470913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.470916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000074_00000_0000000001_00000?, parameters.CurOffset=74
2025-11-30T08:31:14.471082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 74.0/1
2025-11-30T08:31:14.471094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 74 partNo 0
2025-11-30T08:31:14.471106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 74 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 2884149 count 11 nextOffset 75 batches 6
2025-11-30T08:31:14.471108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 74:0 appended
2025-11-30T08:31:14.471112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[11/32] d0000000000_00000000000000000075_00000_0000000001_00000?
2025-11-30T08:31:14.471113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.471116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000075_00000_0000000001_00000?, parameters.CurOffset=75
2025-11-30T08:31:14.471278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 75.0/1
2025-11-30T08:31:14.471290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 75 partNo 0
2025-11-30T08:31:14.473965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 75 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 3146322 count 12 nextOffset 76 batches 6
2025-11-30T08:31:14.473980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 75:0 appended
2025-11-30T08:31:14.473984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[12/32] d0000000000_00000000000000000076_00000_0000000001_00000?
2025-11-30T08:31:14.473985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.473988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000076_00000_0000000001_00000?, parameters.CurOffset=76
2025-11-30T08:31:14.474117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 76.0/1
2025-11-30T08:31:14.474129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 76 partNo 0
2025-11-30T08:31:14.474145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 76 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 3408536 count 13 nextOffset 77 batches 7
2025-11-30T08:31:14.474149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 76:0 appended
2025-11-30T08:31:14.474154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[13/32] d0000000000_00000000000000000077_00000_0000000001_00000?
2025-11-30T08:31:14.474156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.474159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000077_00000_0000000001_00000?, parameters.CurOffset=77
2025-11-30T08:31:14.474281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 77.0/1
2025-11-30T08:31:14.474291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 77 partNo 0
2025-11-30T08:31:14.476916Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 77 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 3670709 count 14 nextOffset 78 batches 7
2025-11-30T08:31:14.476931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 77:0 appended
2025-11-30T08:31:14.476935Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[14/32] d0000000000_00000000000000000078_00000_0000000001_00000?
2025-11-30T08:31:14.476937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.476939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000078_00000_0000000001_00000?, parameters.CurOffset=78
2025-11-30T08:31:14.478208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 78.0/1
2025-11-30T08:31:14.478228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 78 partNo 0
2025-11-30T08:31:14.478250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 78 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 3932923 count 15 nextOffset 79 batches 8
2025-11-30T08:31:14.478260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 78:0 appended
2025-11-30T08:31:14.478264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[15/32] d0000000000_00000000000000000079_00000_0000000001_00000?
2025-11-30T08:31:14.478265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.478267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000079_00000_0000000001_00000?, parameters.CurOffset=79
2025-11-30T08:31:14.480030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 79.0/1
2025-11-30T08:31:14.480049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 79 partNo 0
2025-11-30T08:31:14.482754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 79 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 4195096 count 16 nextOffset 80 batches 8
2025-11-30T08:31:14.482771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 79:0 appended
2025-11-30T08:31:14.482778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[16/32] d0000000000_00000000000000000080_00000_0000000001_00000?
2025-11-30T08:31:14.482782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.482785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000080_00000_0000000001_00000?, parameters.CurOffset=80
2025-11-30T08:31:14.484138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 80.0/1
2025-11-30T08:31:14.484146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 80 partNo 0
2025-11-30T08:31:14.484165Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 80 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 4457310 count 17 nextOffset 81 batches 9
2025-11-30T08:31:14.484167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 80:0 appended
2025-11-30T08:31:14.484173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[17/32] d0000000000_00000000000000000081_00000_0000000001_00000?
2025-11-30T08:31:14.484175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.484179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000081_00000_0000000001_00000?, parameters.CurOffset=81
2025-11-30T08:31:14.485532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 81.0/1
2025-11-30T08:31:14.485548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 81 partNo 0
2025-11-30T08:31:14.488266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 81 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 4719483 count 18 nextOffset 82 batches 9
2025-11-30T08:31:14.488270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 81:0 appended
2025-11-30T08:31:14.488274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[18/32] d0000000000_00000000000000000082_00000_0000000001_00000?
2025-11-30T08:31:14.488276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.488280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000082_00000_0000000001_00000?, parameters.CurOffset=82
2025-11-30T08:31:14.490004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 82.0/1
2025-11-30T08:31:14.490021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 82 partNo 0
2025-11-30T08:31:14.490040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 82 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 4981697 count 19 nextOffset 83 batches 10
2025-11-30T08:31:14.490043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 82:0 appended
2025-11-30T08:31:14.490052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[19/32] d0000000000_00000000000000000083_00000_0000000001_00000?
2025-11-30T08:31:14.490053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.490059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000083_00000_0000000001_00000?, parameters.CurOffset=83
2025-11-30T08:31:14.491387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 83.0/1
2025-11-30T08:31:14.491403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 83 partNo 0
2025-11-30T08:31:14.494054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 83 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 5243870 count 20 nextOffset 84 batches 10
2025-11-30T08:31:14.494073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 83:0 appended
2025-11-30T08:31:14.494080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[20/32] d0000000000_00000000000000000084_00000_0000000001_00000?
2025-11-30T08:31:14.494083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.494086Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000084_00000_0000000001_00000?, parameters.CurOffset=84
2025-11-30T08:31:14.495785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 84.0/1
2025-11-30T08:31:14.495804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 84 partNo 0
2025-11-30T08:31:14.495826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 84 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 5506084 count 21 nextOffset 85 batches 11
2025-11-30T08:31:14.495829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 84:0 appended
2025-11-30T08:31:14.495834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[21/32] d0000000000_00000000000000000085_00000_0000000001_00000?
2025-11-30T08:31:14.495836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.495839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000085_00000_0000000001_00000?, parameters.CurOffset=85
2025-11-30T08:31:14.496000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 85.0/1
2025-11-30T08:31:14.496004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 85 partNo 0
2025-11-30T08:31:14.496486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 85 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 5768257 count 22 nextOffset 86 batches 11
2025-11-30T08:31:14.496490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 85:0 appended
2025-11-30T08:31:14.496496Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[22/32] d0000000000_00000000000000000086_00000_0000000001_00000?
2025-11-30T08:31:14.496498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.496500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000086_00000_0000000001_00000?, parameters.CurOffset=86
2025-11-30T08:31:14.497981Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 86.0/1
2025-11-30T08:31:14.498003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 86 partNo 0
2025-11-30T08:31:14.498032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 86 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 6030471 count 23 nextOffset 87 batches 12
2025-11-30T08:31:14.498040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 86:0 appended
2025-11-30T08:31:14.498045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[23/32] d0000000000_00000000000000000087_00000_0000000001_00000?
2025-11-30T08:31:14.498047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.498080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000087_00000_0000000001_00000?, parameters.CurOffset=87
2025-11-30T08:31:14.499834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 87.0/1
2025-11-30T08:31:14.499865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 87 partNo 0
2025-11-30T08:31:14.500332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 87 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 6292644 count 24 nextOffset 88 batches 12
2025-11-30T08:31:14.500348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 87:0 appended
2025-11-30T08:31:14.500353Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[24/32] d0000000000_00000000000000000088_00000_0000000001_00000?
2025-11-30T08:31:14.500355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.500357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000088_00000_0000000001_00000?, parameters.CurOffset=88
2025-11-30T08:31:14.500502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 88.0/1
2025-11-30T08:31:14.500516Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 88 partNo 0
2025-11-30T08:31:14.500530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 88 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 6554858 count 25 nextOffset 89 batches 13
2025-11-30T08:31:14.500537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 88:0 appended
2025-11-30T08:31:14.500544Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[25/32] d0000000000_00000000000000000089_00000_0000000001_00000?
2025-11-30T08:31:14.500545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.500547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000089_00000_0000000001_00000?, parameters.CurOffset=89
2025-11-30T08:31:14.500591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 89.0/1
2025-11-30T08:31:14.500600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 89 partNo 0
2025-11-30T08:31:14.501082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 89 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 6817031 count 26 nextOffset 90 batches 13
2025-11-30T08:31:14.501095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 89:0 appended
2025-11-30T08:31:14.501099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[26/32] d0000000000_00000000000000000090_00000_0000000001_00000?
2025-11-30T08:31:14.501101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.501103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000090_00000_0000000001_00000?, parameters.CurOffset=90
2025-11-30T08:31:14.501158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 90.0/1
2025-11-30T08:31:14.501169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 90 partNo 0
2025-11-30T08:31:14.501180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 90 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 7079245 count 27 nextOffset 91 batches 14
2025-11-30T08:31:14.501185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 90:0 appended
2025-11-30T08:31:14.501189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[27/32] d0000000000_00000000000000000091_00000_0000000001_00000?
2025-11-30T08:31:14.501191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.501194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000091_00000_0000000001_00000?, parameters.CurOffset=91
2025-11-30T08:31:14.501243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 91.0/1
2025-11-30T08:31:14.501252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 91 partNo 0
2025-11-30T08:31:14.501699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 91 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 7341418 count 28 nextOffset 92 batches 14
2025-11-30T08:31:14.501711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 91:0 appended
2025-11-30T08:31:14.501716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[28/32] d0000000000_00000000000000000092_00000_0000000001_00000?
2025-11-30T08:31:14.501717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.501719Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000092_00000_0000000001_00000?, parameters.CurOffset=92
2025-11-30T08:31:14.501778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 92.0/1
2025-11-30T08:31:14.501788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 92 partNo 0
2025-11-30T08:31:14.501805Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 92 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 7603632 count 29 nextOffset 93 batches 15
2025-11-30T08:31:14.501813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 92:0 appended
2025-11-30T08:31:14.501817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[29/32] d0000000000_00000000000000000093_00000_0000000001_00000?
2025-11-30T08:31:14.501819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.501837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000093_00000_0000000001_00000?, parameters.CurOffset=93
2025-11-30T08:31:14.501894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 93.0/1
2025-11-30T08:31:14.501903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 93 partNo 0
2025-11-30T08:31:14.509345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 93 partNo 0 result is x0000000000_00000000000000000062_00000_0000000031_00000 size 8128009
2025-11-30T08:31:14.509376Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1106: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000062_00000_0000000031_00000 new key d0000000000_00000000000000000062_00000_0000000031_00000 size 8128009 WTime 1764491474509
2025-11-30T08:31:14.509407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 93 partNo 0 FormedBlobsCount 1 NewHead: Offset 93 PartNo 0 PackedSize 262214 count 1 nextOffset 94 batches 1
2025-11-30T08:31:14.509414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 93:0 appended
2025-11-30T08:31:14.509420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[30/32] d0000000000_00000000000000000094_00000_0000000001_00000?
2025-11-30T08:31:14.509423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.509426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000094_00000_0000000001_00000?, parameters.CurOffset=94
2025-11-30T08:31:14.509490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 94.0/1
2025-11-30T08:31:14.509506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 94 partNo 0
2025-11-30T08:31:14.509659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 94 partNo 0 FormedBlobsCount 0 NewHead: Offset 93 PartNo 0 PackedSize 524387 count 2 nextOffset 95 batches 1
2025-11-30T08:31:14.509663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 94:0 appended
2025-11-30T08:31:14.509669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[31/32] d0000000000_00000000000000000095_00000_0000000001_00000?
2025-11-30T08:31:14.509671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-30T08:31:14.509674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000095_00000_0000000001_00000?, parameters.CurOffset=95
2025-11-30T08:31:14.509731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 95.0/1
2025-11-30T08:31:14.509735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 95 partNo 0
2025-11-30T08:31:14.509748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 95 partNo 0 FormedBlobsCount 0 NewHead: Offset 93 PartNo 0 PackedSize 786601 count 3 nextOffset 96 batches 2
2025-11-30T08:31:14.509752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 95:0 appended
2025-11-30T08:31:14.509882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:636: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 93,3 HeadOffset 62 endOffset 64 curOffset 96 d0000000000_00000000000000000093_00000_0000000003_00000| size 786591 WTime 1764491474509
2025-11-30T08:31:14.510054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492508:3735], now have 1 active actors on pipe
2025-11-30T08:31:14.510101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.510106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.510123Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.510178Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 62 partNo 0 count 31 size 8128009
2025-11-30T08:31:14.510186Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 93 partNo 0 count 3 size 786591
2025-11-30T08:31:14.510960Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 4 Topic 'rt3.dc1--topic1' partition 0 user user1 offset 0 partno 0 count 48 size 26214400 endOffset 96 max time lag 0ms effective offset 0
2025-11-30T08:31:14.511189Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 4 added 2 blobs, size 16256018 count 63 last offset 31, current partition end offset: 96
2025-11-30T08:31:14.511244Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 4. Send blob request.
2025-11-30T08:31:14.511261Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 31 parts_count 0 source 1 size 8128009 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:14.511305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 31 partno 0 count 31 parts_count 0 source 1 size 8128009 accessed 0 times before, last time 2025-11-30T08:31:13.000000Z
2025-11-30T08:31:14.511322Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 4. All 2 blobs are from cache.
2025-11-30T08:31:14.511339Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 31 parts 0 suffix '0'
2025-11-30T08:31:14.511344Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 31 parts 0 suffix '0'
2025-11-30T08:31:14.511665Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 2 blobs
2025-11-30T08:31:14.515180Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.518653Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 2 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.521841Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 4 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.525338Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 6 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.528573Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 8 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.531753Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 10 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.535502Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 12 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.538704Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 14 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.541825Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 16 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.544849Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 18 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.547998Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 20 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.551750Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 22 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.555307Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 24 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.559644Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 26 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.564845Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 28 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.567312Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 30 totakecount 31 count 1 size 262182 from pos 0 cbcount 1
2025-11-30T08:31:14.567699Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 31 totakecount 31 count 1 size 262182 from pos 0 cbcount 1
2025-11-30T08:31:14.568265Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 32 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.568887Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 34 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.569431Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 36 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.569949Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 38 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.570466Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 40 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.570987Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 42 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.571502Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 44 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.572041Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 46 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.572136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.572140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.572142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.572144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.572146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.572209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.597692Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 62 count 31 size 8128009 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.597716Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 93 count 3 size 786591 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.597745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.597772Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 31 parts 0 suffix '0' size 8128009
2025-11-30T08:31:14.597807Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 2 parts 0 suffix '124' size 524387
2025-11-30T08:31:14.597858Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 64 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.597862Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 93 partno 0 count 3 parts 0 suffix '124' size 786591
2025-11-30T08:31:14.597985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.597990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.597993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.597996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.598001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.598012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000062_00000_0000000002_00000|
2025-11-30T08:31:14.598019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000064_00000_0000000001_00000?
2025-11-30T08:31:14.598022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000065_00000_0000000001_00000?
2025-11-30T08:31:14.598024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000066_00000_0000000001_00000?
2025-11-30T08:31:14.598027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000067_00000_0000000001_00000?
2025-11-30T08:31:14.598029Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000068_00000_0000000001_00000?
2025-11-30T08:31:14.598032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000069_00000_0000000001_00000?
2025-11-30T08:31:14.598034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000070_00000_0000000001_00000?
2025-11-30T08:31:14.598037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000071_00000_0000000001_00000?
2025-11-30T08:31:14.598039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000072_00000_0000000001_00000?
2025-11-30T08:31:14.598043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000073_00000_0000000001_00000?
2025-11-30T08:31:14.598046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000074_00000_0000000001_00000?
2025-11-30T08:31:14.598072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000075_00000_0000000001_00000?
2025-11-30T08:31:14.598075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000076_00000_0000000001_00000?
2025-11-30T08:31:14.598077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000077_00000_0000000001_00000?
2025-11-30T08:31:14.598079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000078_00000_0000000001_00000?
2025-11-30T08:31:14.598081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000079_00000_0000000001_00000?
2025-11-30T08:31:14.598084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000080_00000_0000000001_00000?
2025-11-30T08:31:14.598086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000081_00000_0000000001_00000?
2025-11-30T08:31:14.598088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000082_00000_0000000001_00000?
2025-11-30T08:31:14.598096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000083_00000_0000000001_00000?
2025-11-30T08:31:14.598106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000084_00000_0000000001_00000?
2025-11-30T08:31:14.598109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000085_00000_0000000001_00000?
2025-11-30T08:31:14.598111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000086_00000_0000000001_00000?
2025-11-30T08:31:14.598113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000087_00000_0000000001_00000?
2025-11-30T08:31:14.598115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000088_00000_0000000001_00000?
2025-11-30T08:31:14.598117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000089_00000_0000000001_00000?
2025-11-30T08:31:14.598119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000090_00000_0000000001_00000?
2025-11-30T08:31:14.598123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000091_00000_0000000001_00000?
2025-11-30T08:31:14.598126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000092_00000_0000000001_00000?
2025-11-30T08:31:14.598129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000093_00000_0000000001_00000?
2025-11-30T08:31:14.598131Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000094_00000_0000000001_00000?
2025-11-30T08:31:14.598134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2683: [72075186224037892][Partition][0][StateIdle] Key deleted: d0000000000_00000000000000000095_00000_0000000001_00000?
2025-11-30T08:31:14.598169Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.598182Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 64 size 262204 cause it's been evicted from L2. Actual L1 size: 36
2025-11-30T08:31:14.598185Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 62 size 524387 cause it's been evicted from L2. Actual L1 size: 35
2025-11-30T08:31:14.598191Z node 2 :PERSQUEUE NOTICE: read.h:372: [72075186224037892][PQCacheProxy]Have to remove new data from cache. Topic rt3.dc1--topic1, tablet id72075186224037892, cookie 0
2025-11-30T08:31:14.598200Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-30T08:31:14.598220Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000062_00000_0000000002_00000|(+) to d0000000000_00000000000000000062_00000_0000000002_00000|(+)
2025-11-30T08:31:14.598222Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000064_00000_0000000001_00000?(+) to d0000000000_00000000000000000064_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598224Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000065_00000_0000000001_00000?(+) to d0000000000_00000000000000000065_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598226Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000066_00000_0000000001_00000?(+) to d0000000000_00000000000000000066_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598228Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000067_00000_0000000001_00000?(+) to d0000000000_00000000000000000067_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598230Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000068_00000_0000000001_00000?(+) to d0000000000_00000000000000000068_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598232Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000069_00000_0000000001_00000?(+) to d0000000000_00000000000000000069_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598240Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000070_00000_0000000001_00000?(+) to d0000000000_00000000000000000070_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598257Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000071_00000_0000000001_00000?(+) to d0000000000_00000000000000000071_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598259Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000072_00000_0000000001_00000?(+) to d0000000000_00000000000000000072_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598260Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000073_00000_0000000001_00000?(+) to d0000000000_00000000000000000073_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598262Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000074_00000_0000000001_00000?(+) to d0000000000_00000000000000000074_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598265Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000075_00000_0000000001_00000?(+) to d0000000000_00000000000000000075_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598267Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000076_00000_0000000001_00000?(+) to d0000000000_00000000000000000076_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598269Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000077_00000_0000000001_00000?(+) to d0000000000_00000000000000000077_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598271Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000078_00000_0000000001_00000?(+) to d0000000000_00000000000000000078_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598273Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000079_00000_0000000001_00000?(+) to d0000000000_00000000000000000079_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598276Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000080_00000_0000000001_00000?(+) to d0000000000_00000000000000000080_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598279Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000081_00000_0000000001_00000?(+) to d0000000000_00000000000000000081_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598280Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000082_00000_0000000001_00000?(+) to d0000000000_00000000000000000082_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598282Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000083_00000_0000000001_00000?(+) to d0000000000_00000000000000000083_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598285Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000084_00000_0000000001_00000?(+) to d0000000000_00000000000000000084_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598287Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000085_00000_0000000001_00000?(+) to d0000000000_00000000000000000085_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598289Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000086_00000_0000000001_00000?(+) to d0000000000_00000000000000000086_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598291Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000087_00000_0000000001_00000?(+) to d0000000000_00000000000000000087_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598293Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000088_00000_0000000001_00000?(+) to d0000000000_00000000000000000088_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598294Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000089_00000_0000000001_00000?(+) to d0000000000_00000000000000000089_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598295Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000090_00000_0000000001_00000?(+) to d0000000000_00000000000000000090_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598297Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000091_00000_0000000001_00000?(+) to d0000000000_00000000000000000091_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598299Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000092_00000_0000000001_00000?(+) to d0000000000_00000000000000000092_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598306Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000093_00000_0000000001_00000?(+) to d0000000000_00000000000000000093_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598308Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000094_00000_0000000001_00000?(+) to d0000000000_00000000000000000094_00000_0000000001_00000?(+)
2025-11-30T08:31:14.598310Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000095_00000_0000000001_00000?(+) to d0000000000_00000000000000000095_00000_0000000001_00000?(+)
2025-11-30T08:31:14.599709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 65 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599726Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 66 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599731Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 67 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599735Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 68 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599741Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 69 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599747Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 70 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 71 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599756Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 72 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599761Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 73 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599765Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 74 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599769Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 75 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599773Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 76 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599777Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 77 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599782Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 78 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599788Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 79 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599793Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 80 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599803Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 81 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599807Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 82 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599811Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 83 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599815Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 84 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599820Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 85 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599824Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 86 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599828Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 87 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 88 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599836Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 89 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 90 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599843Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 91 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599848Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 92 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599862Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 93 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599866Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 94 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599871Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 95 count 1 actorID [2:7578433167081267540:2364]
2025-11-30T08:31:14.599897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2127: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-30T08:31:14.599916Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 65 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.599931Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:572: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-30T08:31:14.599942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.599944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.599946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.599948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.599949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.599956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-30T08:31:14.600077Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 66 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600151Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 67 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600155Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 68 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600160Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 69 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600178Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 70 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600213Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 71 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600228Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 72 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600232Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 73 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600236Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 74 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600241Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 75 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600245Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 76 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600249Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 77 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600253Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 78 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600258Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 79 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600261Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 80 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600265Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 81 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600268Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 82 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600271Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 83 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600275Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 84 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600278Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 85 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600281Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 86 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600284Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 87 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600288Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 88 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600291Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 89 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600294Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 90 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600297Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 91 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600300Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 92 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600303Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 93 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600306Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 94 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.600309Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 95 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-30T08:31:14.605228Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:14.624925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492508:3735] destroyed
CallPersQueueGRPC response:
2025-11-30T08:31:14.672882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.672901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.672904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.672907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.672909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdReadResult {
    MaxOffset: 96
    Result {
      Offset: 0
      Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
CallPersQueueGRPC request to localhost:30859
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdRead {
    ClientId: "user1"
    Offset: 0
    Count: 48
    ReadTimestampMs: 0
  }
}
2025-11-30T08:31:14.721587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7578433174993492566:3762], now have 1 active actors on pipe
2025-11-30T08:31:14.721765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-30T08:31:14.721778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-30T08:31:14.721816Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 5 Topic 'rt3.dc1--topic1' partition 0 user user1 offset 0 partno 0 count 48 size 26214400 endOffset 96 max time lag 0ms effective offset 0
2025-11-30T08:31:14.721847Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 5 added 2 blobs, size 16256018 count 62 last offset 31, current partition end offset: 96
2025-11-30T08:31:14.721853Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 5. Send blob request.
2025-11-30T08:31:14.721863Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 31 parts_count 0 source 1 size 8128009 accessed 1 times before, last time 2025-11-30T08:31:14.000000Z
2025-11-30T08:31:14.721868Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 31 partno 0 count 31 parts_count 0 source 1 size 8128009 accessed 1 times before, last time 2025-11-30T08:31:14.000000Z
2025-11-30T08:31:14.721879Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 5. All 2 blobs are from cache.
2025-11-30T08:31:14.721900Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 31 parts 0 suffix '0'
2025-11-30T08:31:14.721909Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 31 parts 0 suffix '0'
2025-11-30T08:31:14.721917Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 2 blobs
2025-11-30T08:31:14.722163Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.722488Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 2 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.722814Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 4 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.723120Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 6 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.723455Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 8 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.723721Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 10 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.723923Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 12 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.724091Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 14 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.724261Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 16 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.724435Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 18 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.724600Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 20 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.724774Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 22 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.720947Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-30T08:31:14.725042Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 24 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.725240Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 26 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.725421Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 28 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.725506Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 30 totakecount 31 count 1 size 262182 from pos 0 cbcount 1
2025-11-30T08:31:14.725590Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 31 totakecount 31 count 1 size 262182 from pos 0 cbcount 1
2025-11-30T08:31:14.725765Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 32 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.725954Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 34 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.726153Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 36 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.726345Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 38 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.726522Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 40 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.726714Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 42 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.726912Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 44 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.727099Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 46 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-30T08:31:14.727188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-30T08:31:14.737218Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-30T08:31:14.746698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7578433174993492566:3762] destroyed
CallPersQueueGRPC response:
2025-11-30T08:31:14.773410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.773425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.773428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.773431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.773434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdReadResult {
    MaxOffset: 96
    Result {
      Offset: 0
      Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-30T08:31:14.876865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.876886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.876889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.876892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.876895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-30T08:31:14.977157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2303: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-30T08:31:14.977172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.977175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2311: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-30T08:31:14.977178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2362: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-30T08:31:14.977180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Try persist
[[bad]]assertion failed at ydb/services/persqueue_v1/persqueue_ut.cpp:3344, virtual void NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TTestCaseCacheHead::Execute_(NUnitTest::TTestContext &): (info_half1.BlobsFromDisk == 1) failed: (0 != 1) [[rst]]
[[alt1]]TBackTrace::Capture()+28 (0x1514373C)
NUnitTest::NPrivate::RaiseError(char const*, TBasicString<char, std::__y1::char_traits<char>> const&, bool)+128 (0x152FF740)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TTestCaseCacheHead::Execute_(NUnitTest::TTestContext&)+5702 (0x14E78416)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TCurrentTest::Execute()::'lambda'()::operator()() const+71 (0x14F5A6B7)
NUnitTest::TTestBase::Run(std::__y1::function<void ()>, TBasicString<char, std::__y1::char_traits<char>> const&, char const*, bool)+120 (0x15301918)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TCurrentTest::Execute()+436 (0x14F59FB4)
NUnitTest::TTestFactory::Execute()+817 (0x15302061)
NUnitTest::RunMain(int, char**)+3229 (0x153144CD)
??+0 (0x7F5CFA600D90)
__libc_start_main+128 (0x7F5CFA600E40)
_start+41 (0x13F57029)
[[rst]]
