2025-11-27T14:52:17.331387Z node 1 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[1:7577418117183667920:2083];send_to=[0:7307199536658146131:7762515];
2025-11-27T14:52:17.332469Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
2025-11-27T14:52:17.343260Z node 2 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[2:7577418115542628119:2151];send_to=[0:7307199536658146131:7762515];
2025-11-27T14:52:17.343370Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
test_client.cpp: SetPath # /home/runner/.ya/build/build_root/9gmc/00034c/r3tmp/tmp2r8xj9/pdisk_1.dat
2025-11-27T14:52:17.344464Z node 1 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-11-27T14:52:17.348225Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-11-27T14:52:17.390740Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-11-27T14:52:17.392228Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-11-27T14:52:17.412830Z node 1 :IMPORT WARN: schemeshard_import.cpp:307: Table profiles were not loaded
2025-11-27T14:52:17.433021Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-11-27T14:52:17.433062Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-11-27T14:52:17.435888Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Connecting -> Connected
TServer::EnableGrpc on GrpcPort 16480, node 1
2025-11-27T14:52:17.447589Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-11-27T14:52:17.447613Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-11-27T14:52:17.448570Z node 1 :HIVE WARN: hive_impl.cpp:811: HIVE#72057594037968897 Handle TEvInterconnect::TEvNodeConnected, NodeId 2 Cookie 2
2025-11-27T14:52:17.448800Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-11-27T14:52:17.498746Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:190: distributable config is empty, broken or outdated, will use file: /home/runner/.ya/build/build_root/9gmc/00034c/r3tmp/yandexe6MIih.tmp
2025-11-27T14:52:17.498763Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:196: will try to initialize from file: /home/runner/.ya/build/build_root/9gmc/00034c/r3tmp/yandexe6MIih.tmp
2025-11-27T14:52:17.498852Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:202: successfully initialized from file: /home/runner/.ya/build/build_root/9gmc/00034c/r3tmp/yandexe6MIih.tmp
2025-11-27T14:52:17.498885Z node 1 :NET_CLASSIFIER ERROR: net_classifier.cpp:228: got bad distributable configuration
2025-11-27T14:52:17.555080Z INFO: TTestServer started on Port 22541 GrpcPort 16480
TClient is connected to server localhost:22541
PQClient connected to localhost:16480
WaitRootIsUp 'Root'...
TClient::Ls request: Root
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "Root"
    PathId: 1
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 1
    CreateStep: 0
    ParentPathId: 1
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 1
      SubDomainVersion: 0
      SecurityStateVersion: 0
    }
    ChildrenExist: false
  }
  Children {
    Name: ".sys"
    PathId: 18446744073709551615
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 0
    CreateStep: 0
    ParentPathId: 18446744073709551615
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 0
      Pl...
(TRUNCATED)
WaitRootIsUp 'Root' success.
2025-11-27T14:52:17.589297Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpAlterSubDomain, opId: 281474976720657:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_alter_subdomain.cpp:311)
waiting...
waiting...
2025-11-27T14:52:17.608368Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720659:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
waiting...
2025-11-27T14:52:17.613632Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
waiting...
waiting...
2025-11-27T14:52:17.671528Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-11-27T14:52:17.769911Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7577418115542628408:2300], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-27T14:52:17.769958Z node 2 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-27T14:52:17.770149Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7577418115542628420:2303], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-27T14:52:17.770176Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7577418115542628421:2304], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-27T14:52:17.770211Z node 2 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-27T14:52:17.774308Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateResourcePool, opId: 281474976715657:3, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:179)
2025-11-27T14:52:17.785898Z node 2 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [2:7577418115542628424:2305], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976715657 completed, doublechecking }
2025-11-27T14:52:17.878178Z node 2 :TX_PROXY ERROR: schemereq.cpp:593: Actor# [2:7577418115542628452:2173] txid# 281474976715658, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 9], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-11-27T14:52:18.005447Z node 1 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:625: Compilation failed, self: [1:7577418117183668970:2331], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-11-27T14:52:18.005447Z node 2 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:625: Compilation failed, self: [2:7577418115542628466:2309], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-11-27T14:52:18.005628Z node 2 :KQP_SESSION WARN: kqp_session_actor.cpp:2542: SessionId: ydb://session/3?node_id=2&id=ZThjNWNhYTctNmY0ZDk3YWEtZTMxMTIwNS1kNzRhYWNlYg==, ActorId: [2:7577418115542628406:2299], ActorState: ExecuteState, TraceId: 01kb2wszz7fsrepvwbnqznhgp3, ReplyQueryCompileError, status: SCHEME_ERROR, issues: { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } }, remove tx with tx_id: 
2025-11-27T14:52:18.006155Z node 2 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-11-27T14:52:18.008965Z node 1 :KQP_SESSION WARN: kqp_session_actor.cpp:2542: SessionId: ydb://session/3?node_id=1&id=NDA2YzkxYjYtNzRlNDJiYWEtNWNjOWFjYzItMzUzOGU3Njg=, ActorId: [1:7577418117183668928:2323], ActorState: ExecuteState, TraceId: 01kb2wt01n91mym7001k1n9apc, ReplyQueryCompileError, status: SCHEME_ERROR, issues: { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } }, remove tx with tx_id: 
2025-11-27T14:52:18.009081Z node 1 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-11-27T14:52:18.027340Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720662:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
2025-11-27T14:52:18.042820Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720663:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
2025-11-27T14:52:18.070129Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720664:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
=== Init DC: UPSERT INTO `/Root/PQ/Config/V2/Cluster` (name, balancer, local, enabled, weight) VALUES ("dc1", "localhost", true, true, 1000), ("dc2", "dc2.logbroker.yandex.net", false, true, 1000);
=== CheckClustersList. Subcribe to ClusterTracker from [1:7577418121478636701:3034] 
2025-11-27T14:52:18.333966Z node 1 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-11-27T14:52:18.345740Z node 2 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-11-27T14:52:22.331739Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[1:7577418117183667920:2083];send_to=[0:7307199536658146131:7762515];
2025-11-27T14:52:22.331778Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
2025-11-27T14:52:22.343463Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[2:7577418115542628119:2151];send_to=[0:7307199536658146131:7762515];
2025-11-27T14:52:22.343496Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
=== CheckClustersList. Ok
PQ Client: create topic: rt3.dc1--topic1 with 1 partitions
CallPersQueueGRPC request to localhost:16480
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--topic1"
  }
}
2025-11-27T14:52:23.262390Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--topic1, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:16480
MetaRequest {
  CmdCreateTopic {
    Topic: "rt3.dc1--topic1"
    NumPartitions: 1
    Config {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
  }
}
CallPersQueueGRPC response:
Status: 129
ProxyErrorCode: 53
SchemeStatus: 1
FlatTxId {
  TxId: 281474976720671
  SchemeShardTabletId: 72057594046644480
  PathId: 13
}
ErrorCode: OK
AddTopic: rt3.dc1--topic1
===Run query:``DECLARE $version as Int64; DECLARE $path AS Utf8; DECLARE $cluster as Utf8; UPSERT INTO `/Root/PQ/Config/V2/Topics` (path, dc) VALUES ($path, $cluster); UPSERT INTO `/Root/PQ/Config/V2/Versions` (name, version) VALUES ("Topics", $version);`` with topic = topic1, dc = dc1
2025-11-27T14:52:23.271041Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1656: [72075186224037893][] pipe [1:7577418142953473610:3271] connected; active server actors: 1
2025-11-27T14:52:23.271206Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1520: [72075186224037893][rt3.dc1--topic1] updating configuration. Deleted partitions []. Added partitions [0]
2025-11-27T14:52:23.271328Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:971: [72075186224037893][rt3.dc1--topic1] Discovered subdomain [OwnerId: 72057594046644480, LocalPathId: 1] state, outOfSpace = 0 at RB 72075186224037893
2025-11-27T14:52:23.271643Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:140: [72075186224037893][rt3.dc1--topic1] BALANCER INIT DONE for rt3.dc1--topic1: (0, 72075186224037892) 
2025-11-27T14:52:23.277716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2959: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-11-27T14:52:23.278492Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2991: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-11-27T14:52:23.278555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:787: [PQ: 72075186224037892] no config, start with empty partitions and default config
2025-11-27T14:52:23.278578Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:913: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-11-27T14:52:23.278596Z node 2 :PERSQUEUE INFO: pq_impl.cpp:612: [PQ: 72075186224037892] doesn't have tx writes info
2025-11-27T14:52:23.278708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473633:2451], now have 1 active actors on pipe
2025-11-27T14:52:23.278718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473609:3270], now have 1 active actors on pipe
2025-11-27T14:52:23.278917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3563: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-27T14:52:23.279722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-27T14:52:23.281713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473670:3318], now have 1 active actors on pipe
2025-11-27T14:52:23.278761Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:404: [72075186224037893][rt3.dc1--topic1] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1
2025-11-27T14:52:23.281966Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitConfigStep
2025-11-27T14:52:23.282008Z node 2 :PERSQUEUE DEBUG: partition.cpp:1315: [72075186224037892][Partition][0][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-11-27T14:52:23.282077Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-11-27T14:52:23.282165Z node 2 :PERSQUEUE INFO: partition_init.cpp:1107: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7577418141312432507:2363]
2025-11-27T14:52:23.282999Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitFieldsStep
2025-11-27T14:52:23.283004Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--topic1:0:Initializer] Initializing completed.
2025-11-27T14:52:23.283010Z node 2 :PERSQUEUE INFO: partition.cpp:707: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--topic1' partition 0 generation 1 [2:7577418141312432507:2363]
2025-11-27T14:52:23.283026Z node 2 :PERSQUEUE DEBUG: partition.cpp:727: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--topic1 partitition 0 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-11-27T14:52:23.283034Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:107: [72075186224037892][Partition][0][StateIdle] Initializing MLP Consumers: 0
2025-11-27T14:52:23.283381Z node 2 :PERSQUEUE DEBUG: partition.cpp:4404: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 1
2025-11-27T14:52:23.283389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.283407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2392: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TTransaction[ProposeConfig])
2025-11-27T14:52:23.283429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.283431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.283456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.283458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.283479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.283481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.283483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.283485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.283486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.283496Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:930: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-11-27T14:52:23.283503Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:70: [72075186224037892][Partition][0][StateIdle] Process MLP pending events. Count 0
2025-11-27T14:52:23.283585Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3563: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-27T14:52:23.283625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.284486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-27T14:52:23.284539Z node 2 :PERSQUEUE DEBUG: partition.cpp:1420: [72075186224037892][Partition][0][StateIdle] Handle TEvPQ::TEvTxCommit Step 1764255143329, TxId 281474976720671
2025-11-27T14:52:23.284547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.284550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.284551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.284557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2456: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TTransaction[ProposeConfig])
2025-11-27T14:52:23.284576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.284583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.284585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.284625Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.285356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.285396Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:107: [72075186224037892][Partition][0][StateIdle] Initializing MLP Consumers: 0
2025-11-27T14:52:23.285435Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.285447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.285449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.285451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.285453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.285455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.285459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.285564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:398: [PQ: 72075186224037892] Apply new config PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--topic1" Version: 0 LocalDC: true Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } TopicPath: "/Root/PQ/rt3.dc1--topic1" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Version: 0 Generation: 0 } MonitoringProjectId: ""
2025-11-27T14:52:23.285591Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:913: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-11-27T14:52:23.285630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3563: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-27T14:52:23.286275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-27T14:52:23.286313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3563: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-27T14:52:23.287073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
===Query complete
TClient::Ls request: /Root/PQ/rt3.dc1--topic1
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--topic1"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720671
    CreateStep: 1764255143329
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--topic1"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 600000...
(TRUNCATED)
GetTopicVersionFromPath:  record Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--topic1"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720671
    CreateStep: 1764255143329
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--topic1"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      YdbDatabasePath: "/Root"
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
    Partitions {
      PartitionId: 0
      TabletId: 72075186224037892
      Status: Active
    }
    AlterVersion: 1
    BalancerTabletID: 72075186224037893
    NextPartitionId: 1
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 1
      PlanResolution: 7
      Coordinators: 72057594046316545
      TimeCastBucketsPerMediator: 2
      Mediators: 72057594046382081
    }
    DomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    PathsInside: 12
    PathsLimit: 10000
    ShardsInside: 6
    ShardsLimit: 200000
    ResourcesDomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    DiskSpaceUsage {
      Tables {
        TotalSize: 0
        DataSize: 0
        IndexSize: 0
      }
      Topics {
        ReserveSize: 0
        AccountSize: 0
        DataSize: 0
        UsedReserveSize: 0
      }
    }
    PQPartitionsInside: 1
    PQPartitionsLimit: 1000000
    SchemeLimits {
      MaxDepth: 32
      MaxPaths: 10000
      MaxChildrenInDir: 100000
      MaxAclBytesSize: 10240
      MaxTableColumns: 200
      MaxTableColumnNameLength: 255
      MaxTableKeyColumns: 30
      MaxTableIndices: 20
      MaxShards: 200000
      MaxShardsInPath: 35000
      MaxConsistentCopyTargets: 10000
      MaxPathElementLength: 255
      ExtraPathSymbolsAllowed: "!\"#$%&\'()*+,-.:;<=>?@[\\]^_`{|}~"
      MaxPQPartitions: 1000000
      MaxTableCdcStreams: 5
      MaxExports: 10
      MaxImports: 10
      MaxColumnTableColumns: 10000
    }
  }
}
Path: "/Root/PQ/rt3.dc1--topic1"
 name rt3.dc1--topic1 version1
CallPersQueueGRPC request to localhost:16480
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--topic1"
  }
}
2025-11-27T14:52:23.300504Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
MetaResponse {
  CmdGetTopicMetadataResult {
    TopicInfo {
      Topic: "rt3.dc1--topic1"
      NumPartitions: 1
      Config {
        PartitionConfig {
          LifetimeSeconds: 86400
          LowWatermark: 8388608
          SourceIdLifetimeSeconds: 86400
          WriteSpeedInBytesPerSecond: 20000000
          BurstSize: 20000000
          SourceIdMaxCounts: 6000000
        }
        Version: 1
        LocalDC: true
        Codecs {
          Ids: 0
          Ids: 1
          Ids: 2
          Codecs: "raw"
          Codecs: "gzip"
          Codecs: "lzop"
        }
        TopicPath: "/Root/PQ/rt3.dc1--topic1"
        YdbDatabasePath: "/Root"
        Consumers {
          Name: "user"
          ReadFromTimestampsMs: 0
          FormatVersion: 0
          Version: 0
        }
      }
      ErrorCode: OK
    }
  }
}
=== Topic created, have version: 1
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.302371Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.302819Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473730:3353], now have 1 active actors on pipe
2025-11-27T14:52:23.302944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.302966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.303020Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|2e611540-532d07d-4870be32-e7b74fa8_0 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.303037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.303042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.303048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.303056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.303060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.303070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.303072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.303076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.303096Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.303117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.303206Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.303525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473730:3353] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|2e611540-532d07d-4870be32-e7b74fa8_0"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|2e611540-532d07d-4870be32-e7b74fa8_0"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 0
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.315593Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.316200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473733:3354], now have 1 active actors on pipe
2025-11-27T14:52:23.317865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.317887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.318411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 0 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.318844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 0 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.318881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 0 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.318890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 0 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.318918Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.318944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.318953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.318964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.318967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.318972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.318975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.318981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.319026Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 0
2025-11-27T14:52:23.319047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.319055Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 1
2025-11-27T14:52:23.319059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.319062Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 2
2025-11-27T14:52:23.319454Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 0 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1048790 count 1 nextOffset 1 batches 3
2025-11-27T14:52:23.319473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.319476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.319479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.319573Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 0,1 HeadOffset 0 endOffset 0 curOffset 1 d0000000000_00000000000000000000_00000_0000000001_00002? size 1048780 WTime 1764255143319
2025-11-27T14:52:23.320505Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.320533Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.325186Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 0 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.325220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.325244Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.325252Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.325259Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.325274Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 0, partNo: 0, Offset: 0 is stored on disk
2025-11-27T14:52:23.325278Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.325284Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 0, partNo: 1, Offset: 0 is stored on disk
2025-11-27T14:52:23.325289Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.325294Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 0, partNo: 2, Offset: 0 is stored on disk
2025-11-27T14:52:23.325345Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:930: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-11-27T14:52:23.325358Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:972: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-11-27T14:52:23.325363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.325369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.325370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.325373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.325375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.325385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.325408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.325452Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--topic1' partition 0 user user offset 0 partno 0 count 1 size 1024000 endOffset 1 max time lag 0ms effective offset 0
2025-11-27T14:52:23.325474Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 1048780 count 1 last offset 0, current partition end offset: 1
2025-11-27T14:52:23.325481Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-11-27T14:52:23.325505Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.325517Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 3. All 1 blobs are from cache.
2025-11-27T14:52:23.325532Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.325540Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 1 blobs
2025-11-27T14:52:23.325504Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 0
    Offset: 0
    PartNo: 0
    WriteTimestampMS: 1764255143318
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 6
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.326109Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 1 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:23.326253Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 1 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:23.326280Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 1 count 1 size 24704 from pos 0 cbcount 1
2025-11-27T14:52:23.326314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473733:3354] destroyed
2025-11-27T14:52:23.326328Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1022: Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp done, result 1764255143318 queuesize 0 startOffset 0
2025-11-27T14:52:23.326598Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.327056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473740:3358], now have 1 active actors on pipe
2025-11-27T14:52:23.327145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.327159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.327195Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|d3cd46c8-403eb01e-9cede62c-c122b534_1 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.327211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.327219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.327224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.327226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.327229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.327239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.327242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.327245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.327260Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.327272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.327365Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.327597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473740:3358] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|d3cd46c8-403eb01e-9cede62c-c122b534_1"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|d3cd46c8-403eb01e-9cede62c-c122b534_1"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 1
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.338457Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.338991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473747:3361], now have 1 active actors on pipe
2025-11-27T14:52:23.340200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.340219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.340297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 1 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.340741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 1 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.340761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 1 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.340767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 1 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.340796Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.340822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.340830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.340841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.340848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.340853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.340856Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.340861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.340900Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 0
2025-11-27T14:52:23.340919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.340927Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 1
2025-11-27T14:52:23.340931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.340935Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 2
2025-11-27T14:52:23.341550Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 1 partNo 2 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 1048790 count 1 nextOffset 2 batches 3
2025-11-27T14:52:23.341575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.341577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.341580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.341656Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 1,1 HeadOffset 1 endOffset 1 curOffset 2 d0000000000_00000000000000000001_00000_0000000001_00002? size 1048780 WTime 1764255143341
2025-11-27T14:52:23.342013Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.342034Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 1 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.345114Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 1 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.345148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.345176Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.345185Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.345190Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.345201Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 1, partNo: 0, Offset: 1 is stored on disk
2025-11-27T14:52:23.345210Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.345215Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 1, partNo: 1, Offset: 1 is stored on disk
2025-11-27T14:52:23.345221Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.345244Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 1, partNo: 2, Offset: 1 is stored on disk
2025-11-27T14:52:23.345287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.345295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.345298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.345300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.345302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.345313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.345330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.345417Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.345786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473747:3361] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 1
    Offset: 1
    PartNo: 0
    WriteTimestampMS: 1764255143340
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.346538Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.346993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473752:3364], now have 1 active actors on pipe
2025-11-27T14:52:23.347086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.347100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.347140Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|8e6218ce-827ffcc0-df302ba6-72cacf10_2 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.347155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.347159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.347163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.347165Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.347169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.347178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.347186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.347189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.347204Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.347221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.347293Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.347551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473752:3364] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|8e6218ce-827ffcc0-df302ba6-72cacf10_2"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|8e6218ce-827ffcc0-df302ba6-72cacf10_2"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 2
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.354793Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.355287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473755:3365], now have 1 active actors on pipe
2025-11-27T14:52:23.356468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.356506Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.356584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 2 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.356657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 2 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.356672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 2 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.356677Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 2 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.356701Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.356726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.356734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.356742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.356749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.356755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.356757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.356762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.356798Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 0
2025-11-27T14:52:23.356816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.356824Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 1
2025-11-27T14:52:23.356827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.356830Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 2
2025-11-27T14:52:23.357048Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 2 partNo 2 FormedBlobsCount 0 NewHead: Offset 2 PartNo 0 PackedSize 1048790 count 1 nextOffset 3 batches 3
2025-11-27T14:52:23.357069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.357072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.357075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.357147Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 2,1 HeadOffset 2 endOffset 2 curOffset 3 d0000000000_00000000000000000002_00000_0000000001_00002? size 1048780 WTime 1764255143356
2025-11-27T14:52:23.357355Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.357378Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 2 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.360496Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 2 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.360530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.360543Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.360557Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.360564Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.360573Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 2, partNo: 0, Offset: 2 is stored on disk
2025-11-27T14:52:23.360578Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.360583Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 2, partNo: 1, Offset: 2 is stored on disk
2025-11-27T14:52:23.360589Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.360597Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 2, partNo: 2, Offset: 2 is stored on disk
2025-11-27T14:52:23.360636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.360646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.360649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.360651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.360654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.360663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.360680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.360765Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.361107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473755:3365] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 2
    Offset: 2
    PartNo: 0
    WriteTimestampMS: 1764255143356
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.361848Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.362270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473760:3368], now have 1 active actors on pipe
2025-11-27T14:52:23.362380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.362392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.362438Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f3c52a63-2421f2bf-1de63220-4e1b4cb9_3 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.362451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.362455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.362460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.362462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.362465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.362474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.362477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.362480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.362494Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.362513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.362560Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.362727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473760:3368] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f3c52a63-2421f2bf-1de63220-4e1b4cb9_3"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f3c52a63-2421f2bf-1de63220-4e1b4cb9_3"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 3
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.370088Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.370653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473763:3369], now have 1 active actors on pipe
2025-11-27T14:52:23.371821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.371850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.371942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 3 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.372244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 3 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.372267Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 3 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.372273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 3 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.372299Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.372325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.372334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.372344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.372351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.372356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.372359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.372364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.372402Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 0
2025-11-27T14:52:23.372423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.372432Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 1
2025-11-27T14:52:23.372436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.372439Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 2
2025-11-27T14:52:23.372755Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 3 partNo 2 FormedBlobsCount 0 NewHead: Offset 3 PartNo 0 PackedSize 1048790 count 1 nextOffset 4 batches 3
2025-11-27T14:52:23.372779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.372782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.372786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.372861Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 3,1 HeadOffset 3 endOffset 3 curOffset 4 d0000000000_00000000000000000003_00000_0000000001_00002? size 1048780 WTime 1764255143372
2025-11-27T14:52:23.373338Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.373364Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 3 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.376264Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 3 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.376293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.376317Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.376322Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.376329Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.376338Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 3, partNo: 0, Offset: 3 is stored on disk
2025-11-27T14:52:23.376342Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.376346Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 3, partNo: 1, Offset: 3 is stored on disk
2025-11-27T14:52:23.376353Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.376363Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 3, partNo: 2, Offset: 3 is stored on disk
2025-11-27T14:52:23.376400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.376409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.376411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.376414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.376416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.376426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.376447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.376565Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.376957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473763:3369] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 3
    Offset: 3
    PartNo: 0
    WriteTimestampMS: 1764255143372
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.377776Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.378214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473768:3372], now have 1 active actors on pipe
2025-11-27T14:52:23.378321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.378338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.378378Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|da639f40-4f627137-121e056d-4f4eab60_4 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.378393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.378396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.378401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.378403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.378407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.378417Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.378419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.378422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.378437Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.378455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.378505Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.378726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473768:3372] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|da639f40-4f627137-121e056d-4f4eab60_4"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|da639f40-4f627137-121e056d-4f4eab60_4"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 4
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.385943Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.386511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473771:3373], now have 1 active actors on pipe
2025-11-27T14:52:23.387120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.387132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.387140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.387142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.387145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.387733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.387753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.387865Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 4 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.387939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 4 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.387957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 4 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.387963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 4 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.387992Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.388015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.388024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.388034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.388042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.388047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.388050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.388055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.388107Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 0
2025-11-27T14:52:23.388126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.388131Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 1
2025-11-27T14:52:23.388134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.388138Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 2
2025-11-27T14:52:23.388777Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 4 partNo 2 FormedBlobsCount 0 NewHead: Offset 4 PartNo 0 PackedSize 1048790 count 1 nextOffset 5 batches 3
2025-11-27T14:52:23.388800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.388804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.388807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.388886Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 4,1 HeadOffset 4 endOffset 4 curOffset 5 d0000000000_00000000000000000004_00000_0000000001_00002? size 1048780 WTime 1764255143388
2025-11-27T14:52:23.389079Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.389101Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 4 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.392450Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 4 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.392479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.392507Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.392514Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.392516Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.392527Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 4, partNo: 0, Offset: 4 is stored on disk
2025-11-27T14:52:23.392531Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.392535Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 4, partNo: 1, Offset: 4 is stored on disk
2025-11-27T14:52:23.392541Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.392549Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 4, partNo: 2, Offset: 4 is stored on disk
2025-11-27T14:52:23.392586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.392598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.392600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.392602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.392604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.392613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.392631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.392725Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.393119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473771:3373] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 4
    Offset: 4
    PartNo: 0
    WriteTimestampMS: 1764255143387
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.393904Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.394335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473776:3376], now have 1 active actors on pipe
2025-11-27T14:52:23.394527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.394540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.394577Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|de66f417-14cd74b5-50c9b94a-13bb397d_5 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.394593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.394597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.394602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.394604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.394609Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.394618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.394621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.394627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.394639Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.394652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.394708Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.394966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473776:3376] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|de66f417-14cd74b5-50c9b94a-13bb397d_5"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|de66f417-14cd74b5-50c9b94a-13bb397d_5"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 5
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.403382Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.404018Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473779:3377], now have 1 active actors on pipe
2025-11-27T14:52:23.405329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.405359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.405461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 5 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.405538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 5 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.405555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 5 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.405562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 5 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.405590Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.405619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.405628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.405640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.405661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.405674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.405676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.405681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.405723Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 0
2025-11-27T14:52:23.405745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.405749Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 1
2025-11-27T14:52:23.405754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.405758Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 2
2025-11-27T14:52:23.406455Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 5 partNo 2 FormedBlobsCount 0 NewHead: Offset 5 PartNo 0 PackedSize 1048790 count 1 nextOffset 6 batches 3
2025-11-27T14:52:23.406483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.406486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.406489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.406571Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 5,1 HeadOffset 5 endOffset 5 curOffset 6 d0000000000_00000000000000000005_00000_0000000001_00002? size 1048780 WTime 1764255143406
2025-11-27T14:52:23.406798Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.406821Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 5 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.410400Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 5 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.410434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.410450Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.410465Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.410473Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.410489Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 5, partNo: 0, Offset: 5 is stored on disk
2025-11-27T14:52:23.410499Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.410504Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 5, partNo: 1, Offset: 5 is stored on disk
2025-11-27T14:52:23.410509Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.410513Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 5, partNo: 2, Offset: 5 is stored on disk
2025-11-27T14:52:23.410554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.410562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.410565Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.410568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.410570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.410580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.410597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.410705Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.411102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473779:3377] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 5
    Offset: 5
    PartNo: 0
    WriteTimestampMS: 1764255143405
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.411947Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.412368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473784:3380], now have 1 active actors on pipe
2025-11-27T14:52:23.412485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.412499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.412537Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ee7532d3-45b45d99-5cb8d7b8-5d740824_6 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.412554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.412558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.412563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.412565Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.412570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.412580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.412585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.412589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.412603Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.412615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.412667Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.412898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473784:3380] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ee7532d3-45b45d99-5cb8d7b8-5d740824_6"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ee7532d3-45b45d99-5cb8d7b8-5d740824_6"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 6
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.421134Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.421704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473787:3381], now have 1 active actors on pipe
2025-11-27T14:52:23.422880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.422905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.423011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 6 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.423079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 6 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.423097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 6 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.423103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 6 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.423131Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.423159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.423167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.423178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.423186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.423191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.423196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.423201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.423238Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 0
2025-11-27T14:52:23.423256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.423261Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 1
2025-11-27T14:52:23.423264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.423268Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 2
2025-11-27T14:52:23.424126Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 6 partNo 2 FormedBlobsCount 0 NewHead: Offset 6 PartNo 0 PackedSize 1048790 count 1 nextOffset 7 batches 3
2025-11-27T14:52:23.424151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.424154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.424158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.424237Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 6,1 HeadOffset 6 endOffset 6 curOffset 7 d0000000000_00000000000000000006_00000_0000000001_00002? size 1048780 WTime 1764255143424
2025-11-27T14:52:23.424440Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.424466Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 6 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.428324Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 6 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.428357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.428389Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.428396Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.428408Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 6, partNo: 0, Offset: 6 is stored on disk
2025-11-27T14:52:23.428419Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.428424Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 6, partNo: 1, Offset: 6 is stored on disk
2025-11-27T14:52:23.428430Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.428434Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.428440Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 6, partNo: 2, Offset: 6 is stored on disk
2025-11-27T14:52:23.428481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.428510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.428518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.428520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.428523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.428533Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.428553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.428651Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 6
    Offset: 6
    PartNo: 0
    WriteTimestampMS: 1764255143423
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.429631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473787:3381] destroyed
2025-11-27T14:52:23.431937Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.432435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473792:3384], now have 1 active actors on pipe
2025-11-27T14:52:23.432590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.432603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.432641Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|886bee6c-fc4b7d8c-920ab704-5465d4c0_7 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.432653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.432657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.432662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.432665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.432669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.432683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.432687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.432690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.432708Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.432723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.432798Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.433154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473792:3384] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|886bee6c-fc4b7d8c-920ab704-5465d4c0_7"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|886bee6c-fc4b7d8c-920ab704-5465d4c0_7"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 7
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.443520Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.444112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473795:3385], now have 1 active actors on pipe
2025-11-27T14:52:23.445462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.445489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.445577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 7 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.445640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 7 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.445650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 7 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.445657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 7 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.445684Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.445708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.445713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.445723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.445726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.445733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.445736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.445740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.445778Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 0
2025-11-27T14:52:23.445793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.445796Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 1
2025-11-27T14:52:23.445800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.445803Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 2
2025-11-27T14:52:23.446644Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 7 partNo 2 FormedBlobsCount 0 NewHead: Offset 7 PartNo 0 PackedSize 1048790 count 1 nextOffset 8 batches 3
2025-11-27T14:52:23.446671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.446675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.446679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.446764Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 7,1 HeadOffset 7 endOffset 7 curOffset 8 d0000000000_00000000000000000007_00000_0000000001_00002? size 1048780 WTime 1764255143446
2025-11-27T14:52:23.446975Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.446990Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 7 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.455243Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 7 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.455280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.455308Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.455312Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.455321Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.455333Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 7, partNo: 0, Offset: 7 is stored on disk
2025-11-27T14:52:23.455339Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.455344Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 7, partNo: 1, Offset: 7 is stored on disk
2025-11-27T14:52:23.455352Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.455357Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 7, partNo: 2, Offset: 7 is stored on disk
2025-11-27T14:52:23.455405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.455409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.455411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.455414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.455416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.455428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000000_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.455431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000001_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.455452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000002_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.455455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000003_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.455458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000004_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.455461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000005_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.455465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000006_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.455467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000007_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.455470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 8 keys were taken away. Let's read 8390240 bytes
2025-11-27T14:52:23.455491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.455509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 8 blobs
2025-11-27T14:52:23.455513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000000_00000_0000000001_00002?
2025-11-27T14:52:23.455527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000001_00000_0000000001_00002?
2025-11-27T14:52:23.455542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000002_00000_0000000001_00002?
2025-11-27T14:52:23.455560Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000003_00000_0000000001_00002?
2025-11-27T14:52:23.455569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000004_00000_0000000001_00002?
2025-11-27T14:52:23.455600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000005_00000_0000000001_00002?
2025-11-27T14:52:23.455612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000006_00000_0000000001_00002?
2025-11-27T14:52:23.455624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000007_00000_0000000001_00002?
2025-11-27T14:52:23.455635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 8 blobs for compaction
2025-11-27T14:52:23.455647Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 1 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.455652Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 1 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.455656Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 2 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.455661Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 3 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.455665Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 4 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.455669Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 5 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.455673Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 6 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.455676Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 7 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.455691Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 8 blobs are from cache.
2025-11-27T14:52:23.455747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-27T14:52:23.455758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/8] d0000000000_00000000000000000000_00000_0000000001_00002?
2025-11-27T14:52:23.455761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.455765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000000_00000_0000000001_00002?, parameters.CurOffset=0
2025-11-27T14:52:23.455903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 0.0/3
2025-11-27T14:52:23.455913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 0
2025-11-27T14:52:23.455920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 0:0 appended
2025-11-27T14:52:23.456076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 0.1/3
2025-11-27T14:52:23.456081Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 1
2025-11-27T14:52:23.456084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 0:1 appended
2025-11-27T14:52:23.456102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 0.2/3
2025-11-27T14:52:23.456106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 2
2025-11-27T14:52:23.456348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 0 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1048790 count 1 nextOffset 1 batches 3
2025-11-27T14:52:23.456355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 0:2 appended
2025-11-27T14:52:23.456359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/8] d0000000000_00000000000000000001_00000_0000000001_00002?
2025-11-27T14:52:23.456362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.456365Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000001_00000_0000000001_00002?, parameters.CurOffset=1
2025-11-27T14:52:23.456509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 1.0/3
2025-11-27T14:52:23.456514Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 0
2025-11-27T14:52:23.456517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 1:0 appended
2025-11-27T14:52:23.456856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 1.1/3
2025-11-27T14:52:23.456863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 1
2025-11-27T14:52:23.456866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 1:1 appended
2025-11-27T14:52:23.456906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 1.2/3
2025-11-27T14:52:23.456909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 2
2025-11-27T14:52:23.457121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 1 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2097545 count 2 nextOffset 2 batches 5
2025-11-27T14:52:23.457128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 1:2 appended
2025-11-27T14:52:23.457133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/8] d0000000000_00000000000000000002_00000_0000000001_00002?
2025-11-27T14:52:23.457136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.457140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000002_00000_0000000001_00002?, parameters.CurOffset=2
2025-11-27T14:52:23.457275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 2.0/3
2025-11-27T14:52:23.457280Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 0
2025-11-27T14:52:23.457285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 2:0 appended
2025-11-27T14:52:23.457785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 2.1/3
2025-11-27T14:52:23.457790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 1
2025-11-27T14:52:23.457793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 2:1 appended
2025-11-27T14:52:23.457808Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 2.2/3
2025-11-27T14:52:23.457811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 2
2025-11-27T14:52:23.458394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 2 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3146300 count 3 nextOffset 3 batches 7
2025-11-27T14:52:23.458414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 2:2 appended
2025-11-27T14:52:23.458418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/8] d0000000000_00000000000000000003_00000_0000000001_00002?
2025-11-27T14:52:23.458420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.458423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000003_00000_0000000001_00002?, parameters.CurOffset=3
2025-11-27T14:52:23.458500Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.458520Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.458525Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.458529Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.458532Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.458536Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.458539Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.458543Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.458583Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 3.0/3
2025-11-27T14:52:23.458587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 0
2025-11-27T14:52:23.458590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 3:0 appended
2025-11-27T14:52:23.458734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 3.1/3
2025-11-27T14:52:23.458752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 1
2025-11-27T14:52:23.458754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 3:1 appended
2025-11-27T14:52:23.458767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 3.2/3
2025-11-27T14:52:23.458770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 2
2025-11-27T14:52:23.459342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 3 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4195055 count 4 nextOffset 4 batches 9
2025-11-27T14:52:23.459358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 3:2 appended
2025-11-27T14:52:23.459364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/8] d0000000000_00000000000000000004_00000_0000000001_00002?
2025-11-27T14:52:23.459366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.459369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000004_00000_0000000001_00002?, parameters.CurOffset=4
2025-11-27T14:52:23.459528Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 7
    Offset: 7
    PartNo: 0
    WriteTimestampMS: 1764255143445
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 9
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.460114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 4.0/3
2025-11-27T14:52:23.460130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 0
2025-11-27T14:52:23.460134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 4:0 appended
2025-11-27T14:52:23.460265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 4.1/3
2025-11-27T14:52:23.460272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 1
2025-11-27T14:52:23.460275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 4:1 appended
2025-11-27T14:52:23.460286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 4.2/3
2025-11-27T14:52:23.460289Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 2
2025-11-27T14:52:23.460473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 4 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 5243810 count 5 nextOffset 5 batches 11
2025-11-27T14:52:23.460479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 4:2 appended
2025-11-27T14:52:23.460483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/8] d0000000000_00000000000000000005_00000_0000000001_00002?
2025-11-27T14:52:23.460485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.460488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000005_00000_0000000001_00002?, parameters.CurOffset=5
2025-11-27T14:52:23.460623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 5.0/3
2025-11-27T14:52:23.460629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 0
2025-11-27T14:52:23.460631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 5:0 appended
2025-11-27T14:52:23.461141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 5.1/3
2025-11-27T14:52:23.461147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 1
2025-11-27T14:52:23.461151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 5:1 appended
2025-11-27T14:52:23.461163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 5.2/3
2025-11-27T14:52:23.461166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 2
2025-11-27T14:52:23.461726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 5 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 6292565 count 6 nextOffset 6 batches 13
2025-11-27T14:52:23.461734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 5:2 appended
2025-11-27T14:52:23.461738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/8] d0000000000_00000000000000000006_00000_0000000001_00002?
2025-11-27T14:52:23.461740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.461744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000006_00000_0000000001_00002?, parameters.CurOffset=6
2025-11-27T14:52:23.461891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 6.0/3
2025-11-27T14:52:23.461898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 0
2025-11-27T14:52:23.461901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 6:0 appended
2025-11-27T14:52:23.462037Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 6.1/3
2025-11-27T14:52:23.462043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 1
2025-11-27T14:52:23.462045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 6:1 appended
2025-11-27T14:52:23.462057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 6.2/3
2025-11-27T14:52:23.462059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 2
2025-11-27T14:52:23.462624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 6 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 7341320 count 7 nextOffset 7 batches 15
2025-11-27T14:52:23.462631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 6:2 appended
2025-11-27T14:52:23.462635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/8] d0000000000_00000000000000000007_00000_0000000001_00002?
2025-11-27T14:52:23.462636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.462640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000007_00000_0000000001_00002?, parameters.CurOffset=7
2025-11-27T14:52:23.462769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 7.0/3
2025-11-27T14:52:23.462776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 0
2025-11-27T14:52:23.462778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 7:0 appended
2025-11-27T14:52:23.462917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 7.1/3
2025-11-27T14:52:23.462922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 1
2025-11-27T14:52:23.462925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 7:1 appended
2025-11-27T14:52:23.462937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 7.2/3
2025-11-27T14:52:23.462940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 2
2025-11-27T14:52:23.464030Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.472911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 7 partNo 2 result is x0000000000_00000000000000000000_00000_0000000007_00016 size 8365339
2025-11-27T14:52:23.472954Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1104: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000000_00000_0000000007_00016 new key d0000000000_00000000000000000000_00000_0000000007_00016 size 8365339 WTime 1764255143472
2025-11-27T14:52:23.473005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 7 partNo 2 FormedBlobsCount 1 NewHead: Offset 7 PartNo 2 PackedSize 24736 count 1 nextOffset 8 batches 1
2025-11-27T14:52:23.473015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 7:2 appended
2025-11-27T14:52:23.473074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 7,1 HeadOffset 0 endOffset 0 curOffset 8 d0000000000_00000000000000000007_00002_0000000001_00000| size 24726 WTime 1764255143472
2025-11-27T14:52:23.473144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473795:3385] destroyed
2025-11-27T14:52:23.473161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473804:3389], now have 1 active actors on pipe
2025-11-27T14:52:23.473188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.473197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.473213Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.473232Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 7 size 8365339
2025-11-27T14:52:23.473240Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 7 partNo 2 count 1 size 24726
2025-11-27T14:52:23.473286Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|5a5e4ada-1ccaed09-83a5b21-db0180a4_8 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.473306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.473309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.473315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.473317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.473321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.473333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.473336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.473338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.473356Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.473415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.473735Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.474195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473804:3389] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|5a5e4ada-1ccaed09-83a5b21-db0180a4_8"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|5a5e4ada-1ccaed09-83a5b21-db0180a4_8"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 8
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.481907Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.482525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473809:3391], now have 1 active actors on pipe
2025-11-27T14:52:23.483959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.483982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.484072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 8 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.484134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 8 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.484144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 8 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.484150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 8 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.484176Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.484195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.484199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.484209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.484211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.484216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.484218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.484223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.484256Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 0
2025-11-27T14:52:23.484270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.484274Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 1
2025-11-27T14:52:23.484277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.484280Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 2
2025-11-27T14:52:23.484543Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 8 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 1048790 count 1 nextOffset 9 batches 3
2025-11-27T14:52:23.484572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.484576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.484580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.484646Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 8,1 HeadOffset 8 endOffset 8 curOffset 9 d0000000000_00000000000000000008_00000_0000000001_00002? size 1048780 WTime 1764255143484
2025-11-27T14:52:23.484915Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.484940Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 8 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.484959Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 0 count 7 size 8365339 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.484972Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 7 count 1 size 24726 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.485024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.485031Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 7 parts 16 suffix '0' size 8365339
2025-11-27T14:52:23.485038Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 7 partno 2 count 1 parts 0 suffix '124' size 24726
2025-11-27T14:52:23.485066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-27T14:52:23.485075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.487350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-27T14:52:23.488123Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 8 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.488153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.488175Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.488179Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.488185Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.488196Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 8, partNo: 0, Offset: 8 is stored on disk
2025-11-27T14:52:23.488208Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.488212Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 8, partNo: 1, Offset: 8 is stored on disk
2025-11-27T14:52:23.488218Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.488223Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 8, partNo: 2, Offset: 8 is stored on disk
2025-11-27T14:52:23.488275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.488286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.488288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.488290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.488294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.488335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.488360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.488374Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.488390Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000000_00000_0000000001_00002?(+) to d0000000000_00000000000000000000_00000_0000000001_00002?(+)
2025-11-27T14:52:23.488393Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000001_00000_0000000001_00002?(+) to d0000000000_00000000000000000001_00000_0000000001_00002?(+)
2025-11-27T14:52:23.488396Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000002_00000_0000000001_00002?(+) to d0000000000_00000000000000000002_00000_0000000001_00002?(+)
2025-11-27T14:52:23.488398Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000003_00000_0000000001_00002?(+) to d0000000000_00000000000000000003_00000_0000000001_00002?(+)
2025-11-27T14:52:23.488401Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000004_00000_0000000001_00002?(+) to d0000000000_00000000000000000004_00000_0000000001_00002?(+)
2025-11-27T14:52:23.488404Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000005_00000_0000000001_00002?(+) to d0000000000_00000000000000000005_00000_0000000001_00002?(+)
2025-11-27T14:52:23.488407Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000006_00000_0000000001_00002?(+) to d0000000000_00000000000000000006_00000_0000000001_00002?(+)
2025-11-27T14:52:23.488409Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000007_00000_0000000001_00002?(+) to d0000000000_00000000000000000007_00000_0000000001_00002?(+)
2025-11-27T14:52:23.488451Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 8
    Offset: 8
    PartNo: 0
    WriteTimestampMS: 1764255143484
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-27T14:52:23.488893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473809:3391] destroyed
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.489289Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 0 count 1 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.489303Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 1 count 1 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.489309Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 2 count 1 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.489313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 3 count 1 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.489319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 4 count 1 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.489324Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 5 count 1 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.489328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 6 count 1 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.489335Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 7 count 1 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.489353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.489360Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.489369Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.489373Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.489379Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.489382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.489384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.489386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.489387Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.489388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.489390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.489394Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.489397Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.489400Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.489408Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.489416Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.489540Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.489941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473817:3397], now have 1 active actors on pipe
2025-11-27T14:52:23.490175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.490191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.490222Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|2d971d58-24680d2e-86493105-3cf277d1_9 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.490237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.490241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.490246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.490248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.490253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.490262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.490265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.490268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.490281Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.490302Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.490380Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.490680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473817:3397] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|2d971d58-24680d2e-86493105-3cf277d1_9"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|2d971d58-24680d2e-86493105-3cf277d1_9"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 9
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.496950Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.497503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473821:3399], now have 1 active actors on pipe
2025-11-27T14:52:23.498635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.498662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.498876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 9 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.499347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 9 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.499371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 9 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.499376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 9 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.499403Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.499423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.499451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.499460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.499463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.499468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.499470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.499475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.499508Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 0
2025-11-27T14:52:23.499522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.499526Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 1
2025-11-27T14:52:23.499529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.499532Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 2
2025-11-27T14:52:23.499808Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 9 partNo 2 FormedBlobsCount 0 NewHead: Offset 9 PartNo 0 PackedSize 1048790 count 1 nextOffset 10 batches 3
2025-11-27T14:52:23.499834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.499838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.499841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.499921Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 9,1 HeadOffset 9 endOffset 9 curOffset 10 d0000000000_00000000000000000009_00000_0000000001_00002? size 1048780 WTime 1764255143499
2025-11-27T14:52:23.500123Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.500136Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 9 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.503672Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 9 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.503712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.503733Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.503746Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.503755Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.503768Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 9, partNo: 0, Offset: 9 is stored on disk
2025-11-27T14:52:23.503773Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.503778Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 9, partNo: 1, Offset: 9 is stored on disk
2025-11-27T14:52:23.503784Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.503793Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 9, partNo: 2, Offset: 9 is stored on disk
2025-11-27T14:52:23.503832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.503835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.503837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.503840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.503842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.503853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.503871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.503965Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.504889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473821:3399] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 9
    Offset: 9
    PartNo: 0
    WriteTimestampMS: 1764255143499
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.507181Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.507648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473826:3402], now have 1 active actors on pipe
2025-11-27T14:52:23.507778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.507791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.507838Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9f4e31e9-3dafaedf-3fc23887-bc0f0b88_10 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.507850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.507857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.507863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.507865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.507870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.507883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.507887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.507889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.507921Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.507939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.508018Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9f4e31e9-3dafaedf-3fc23887-bc0f0b88_10"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16480
2025-11-27T14:52:23.508567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473826:3402] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9f4e31e9-3dafaedf-3fc23887-bc0f0b88_10"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 10
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.515728Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.516375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473829:3403], now have 1 active actors on pipe
2025-11-27T14:52:23.517539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.517567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.517659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 10 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.517723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 10 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.517733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 10 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.517739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 10 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.517760Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.517790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.517795Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.517807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.517809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.517813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.517816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.517822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.517863Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 0
2025-11-27T14:52:23.517878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.517882Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 1
2025-11-27T14:52:23.517886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.517889Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 2
2025-11-27T14:52:23.518212Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 10 partNo 2 FormedBlobsCount 0 NewHead: Offset 10 PartNo 0 PackedSize 1048790 count 1 nextOffset 11 batches 3
2025-11-27T14:52:23.518224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.518227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.518231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.518307Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 10,1 HeadOffset 10 endOffset 10 curOffset 11 d0000000000_00000000000000000010_00000_0000000001_00002? size 1048780 WTime 1764255143517
2025-11-27T14:52:23.518489Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.518505Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 10 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.522197Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 10 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.522232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.522260Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.522260Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.522271Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.522281Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 10, partNo: 0, Offset: 10 is stored on disk
2025-11-27T14:52:23.522286Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.522291Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 10, partNo: 1, Offset: 10 is stored on disk
2025-11-27T14:52:23.522296Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.522301Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 10, partNo: 2, Offset: 10 is stored on disk
2025-11-27T14:52:23.522342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.522351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.522354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.522356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.522358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.522367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.522384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.522486Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.522805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473829:3403] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 10
    Offset: 10
    PartNo: 0
    WriteTimestampMS: 1764255143517
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.523676Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.524150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473834:3406], now have 1 active actors on pipe
2025-11-27T14:52:23.524448Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|d78a1429-5f247509-adef57d0-b57c5ee4_11"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16480
2025-11-27T14:52:23.524236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.524246Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.524289Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|d78a1429-5f247509-adef57d0-b57c5ee4_11 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.524301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.524304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.524316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.524318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.524321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.524331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.524334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.524336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.524350Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.524363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.524679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473834:3406] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|d78a1429-5f247509-adef57d0-b57c5ee4_11"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 11
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.531324Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.531965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473837:3407], now have 1 active actors on pipe
2025-11-27T14:52:23.533111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.533136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.533220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 11 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.533307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 11 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.533324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 11 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.533329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 11 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.533353Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.533380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.533389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.533400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.533408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.533413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.533415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.533421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.533457Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 0
2025-11-27T14:52:23.533478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.533486Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 1
2025-11-27T14:52:23.533490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.533493Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 2
2025-11-27T14:52:23.533789Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 11 partNo 2 FormedBlobsCount 0 NewHead: Offset 11 PartNo 0 PackedSize 1048790 count 1 nextOffset 12 batches 3
2025-11-27T14:52:23.533807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.533810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.533813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.533893Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 11,1 HeadOffset 11 endOffset 11 curOffset 12 d0000000000_00000000000000000011_00000_0000000001_00002? size 1048780 WTime 1764255143532
2025-11-27T14:52:23.534068Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.534090Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 11 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.546669Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 11 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.546710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.546732Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.546746Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.546756Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.546769Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 11, partNo: 0, Offset: 11 is stored on disk
2025-11-27T14:52:23.546779Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.546784Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 11, partNo: 1, Offset: 11 is stored on disk
2025-11-27T14:52:23.546791Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.546799Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 11, partNo: 2, Offset: 11 is stored on disk
2025-11-27T14:52:23.546847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.546857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.546859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.546862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.546864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.546891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.546917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.547078Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.547679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473837:3407] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 11
    Offset: 11
    PartNo: 0
    WriteTimestampMS: 1764255143532
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 13
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.548542Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.549033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473842:3410], now have 1 active actors on pipe
2025-11-27T14:52:23.549135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.549152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.549194Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|dd7508f6-8c283445-6ebfb066-25e9fe9f_12 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.549210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.549214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.549220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.549222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.549227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.549244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.549252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.549255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.549271Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.549289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.549357Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.549664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473842:3410] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|dd7508f6-8c283445-6ebfb066-25e9fe9f_12"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|dd7508f6-8c283445-6ebfb066-25e9fe9f_12"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 12
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.556504Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.557053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473845:3411], now have 1 active actors on pipe
2025-11-27T14:52:23.558414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.558435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.558528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 12 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.558601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 12 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.558612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 12 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.558617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 12 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.558647Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.558670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.558675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.558686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.558688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.558693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.558695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.558701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.558737Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 0
2025-11-27T14:52:23.558754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.558758Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 1
2025-11-27T14:52:23.558761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.558765Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 2
2025-11-27T14:52:23.559028Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 12 partNo 2 FormedBlobsCount 0 NewHead: Offset 12 PartNo 0 PackedSize 1048790 count 1 nextOffset 13 batches 3
2025-11-27T14:52:23.559059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.559063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.559066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.559149Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 12,1 HeadOffset 12 endOffset 12 curOffset 13 d0000000000_00000000000000000012_00000_0000000001_00002? size 1048780 WTime 1764255143558
2025-11-27T14:52:23.559371Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.559398Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 12 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.563311Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 12 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.563344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.563377Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.563385Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.563396Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 12, partNo: 0, Offset: 12 is stored on disk
2025-11-27T14:52:23.563405Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.563410Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 12, partNo: 1, Offset: 12 is stored on disk
2025-11-27T14:52:23.563418Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.563422Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 12, partNo: 2, Offset: 12 is stored on disk
2025-11-27T14:52:23.563483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.563493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.563495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.563497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.563499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.563511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.563530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.563601Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.563708Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.564200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473845:3411] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 12
    Offset: 12
    PartNo: 0
    WriteTimestampMS: 1764255143558
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.565572Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.566533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473850:3414], now have 1 active actors on pipe
2025-11-27T14:52:23.566692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.566704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.566740Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|91fec19a-984ba348-ea3252b3-1183fc4a_13 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.566753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.566757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.566763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.566765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.566770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.566783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.566786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.566790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.566807Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.566845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.567015Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|91fec19a-984ba348-ea3252b3-1183fc4a_13"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16480
2025-11-27T14:52:23.568787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473850:3414] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|91fec19a-984ba348-ea3252b3-1183fc4a_13"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 13
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.574133Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.575728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473853:3415], now have 1 active actors on pipe
2025-11-27T14:52:23.577173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.577193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.577269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 13 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.577317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 13 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.577330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 13 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.577335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 13 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.577352Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.577376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.577380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.577392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.577395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.577399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.577401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.577406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.577442Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 0
2025-11-27T14:52:23.577460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.577464Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 1
2025-11-27T14:52:23.577468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.577472Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 2
2025-11-27T14:52:23.577815Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 13 partNo 2 FormedBlobsCount 0 NewHead: Offset 13 PartNo 0 PackedSize 1048790 count 1 nextOffset 14 batches 3
2025-11-27T14:52:23.577836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.577839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.577843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.577923Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 13,1 HeadOffset 13 endOffset 13 curOffset 14 d0000000000_00000000000000000013_00000_0000000001_00002? size 1048780 WTime 1764255143577
2025-11-27T14:52:23.578960Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.578982Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 13 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.583610Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 13 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.583646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.583660Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.583676Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.583685Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.583695Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 13, partNo: 0, Offset: 13 is stored on disk
2025-11-27T14:52:23.583700Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.583705Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 13, partNo: 1, Offset: 13 is stored on disk
2025-11-27T14:52:23.583711Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.583719Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 13, partNo: 2, Offset: 13 is stored on disk
2025-11-27T14:52:23.583756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.583760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.583763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.583765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.583768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.583777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.583793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.583891Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.584461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473853:3415] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 13
    Offset: 13
    PartNo: 0
    WriteTimestampMS: 1764255143577
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 6
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.585172Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.586020Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|c7c7402f-23936538-521ff44c-a5a81136_14"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16480
2025-11-27T14:52:23.585653Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473858:3418], now have 1 active actors on pipe
2025-11-27T14:52:23.585829Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.585840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.585887Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|c7c7402f-23936538-521ff44c-a5a81136_14 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.585900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.585904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.585910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.585912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.585917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.585928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.585931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.585934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.585953Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.585970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.586245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473858:3418] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|c7c7402f-23936538-521ff44c-a5a81136_14"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 14
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.587599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.587614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.587619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.587622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.587624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.592001Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.592446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473861:3419], now have 1 active actors on pipe
2025-11-27T14:52:23.593796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.593823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.593968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 14 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.594080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 14 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.594091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 14 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.594096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 14 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.594113Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.594133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.594136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.594147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.594150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.594155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.594158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.594163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.594197Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 0
2025-11-27T14:52:23.594212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.594218Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 1
2025-11-27T14:52:23.594221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.594225Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 2
2025-11-27T14:52:23.594644Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 14 partNo 2 FormedBlobsCount 0 NewHead: Offset 14 PartNo 0 PackedSize 1048790 count 1 nextOffset 15 batches 3
2025-11-27T14:52:23.594668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.594671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.594674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.594750Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 14,1 HeadOffset 14 endOffset 14 curOffset 15 d0000000000_00000000000000000014_00000_0000000001_00002? size 1048780 WTime 1764255143594
2025-11-27T14:52:23.595629Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.595651Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 14 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.599719Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 14 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.599743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.599769Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.599770Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.599777Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.599789Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 14, partNo: 0, Offset: 14 is stored on disk
2025-11-27T14:52:23.599793Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.599798Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 14, partNo: 1, Offset: 14 is stored on disk
2025-11-27T14:52:23.599804Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.599808Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 14, partNo: 2, Offset: 14 is stored on disk
2025-11-27T14:52:23.599839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.599843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.599845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.599847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.599849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.599858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.599873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.599947Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.600322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473861:3419] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 14
    Offset: 14
    PartNo: 0
    WriteTimestampMS: 1764255143593
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.601196Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.601587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473866:3422], now have 1 active actors on pipe
2025-11-27T14:52:23.601689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.601700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.601727Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9df2420b-efcbeb96-61b6b21e-af5dc0c2_15 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.601739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.601743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.601747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.601750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.601754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.601764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.601774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.601779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.601791Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.601803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.601894Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.602274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473866:3422] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9df2420b-efcbeb96-61b6b21e-af5dc0c2_15"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9df2420b-efcbeb96-61b6b21e-af5dc0c2_15"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 15
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.609778Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.610299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473869:3423], now have 1 active actors on pipe
2025-11-27T14:52:23.611513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.611531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.611671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 15 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.611758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 15 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.611769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 15 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.611773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 15 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.611795Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.611817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.611821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.611832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.611834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.611839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.611841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.611846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.611878Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 0
2025-11-27T14:52:23.611892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.611895Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 1
2025-11-27T14:52:23.611899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.611902Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 2
2025-11-27T14:52:23.612349Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 15 partNo 2 FormedBlobsCount 0 NewHead: Offset 15 PartNo 0 PackedSize 1048790 count 1 nextOffset 16 batches 3
2025-11-27T14:52:23.612376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.612379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.612382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.612459Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 15,1 HeadOffset 15 endOffset 15 curOffset 16 d0000000000_00000000000000000015_00000_0000000001_00002? size 1048780 WTime 1764255143612
2025-11-27T14:52:23.612630Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.612654Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 15 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.615909Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 15 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.615939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.615963Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.615965Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.615971Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.615982Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 15, partNo: 0, Offset: 15 is stored on disk
2025-11-27T14:52:23.615986Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.615991Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 15, partNo: 1, Offset: 15 is stored on disk
2025-11-27T14:52:23.615998Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.616006Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 15, partNo: 2, Offset: 15 is stored on disk
2025-11-27T14:52:23.616048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.616069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.616071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.616074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.616076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.616087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000008_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.616091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000009_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.616094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000010_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.616096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000011_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.616100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000012_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.616102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000013_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.616105Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000014_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.616107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000015_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.616110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 8 keys were taken away. Let's read 8390240 bytes
2025-11-27T14:52:23.616129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.616149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 8 blobs
2025-11-27T14:52:23.616157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000008_00000_0000000001_00002?
2025-11-27T14:52:23.616168Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000009_00000_0000000001_00002?
2025-11-27T14:52:23.616184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000010_00000_0000000001_00002?
2025-11-27T14:52:23.616202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000011_00000_0000000001_00002?
2025-11-27T14:52:23.616216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000012_00000_0000000001_00002?
2025-11-27T14:52:23.616234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000013_00000_0000000001_00002?
2025-11-27T14:52:23.616250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000014_00000_0000000001_00002?
2025-11-27T14:52:23.616259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000015_00000_0000000001_00002?
2025-11-27T14:52:23.616274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 8 blobs for compaction
2025-11-27T14:52:23.616286Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 8 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.616295Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 9 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.616300Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 10 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.616304Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 11 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.616308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 12 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.616311Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 13 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.616316Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 14 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.616319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 15 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.616336Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 8 blobs are from cache.
2025-11-27T14:52:23.616371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-27T14:52:23.616386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/8] d0000000000_00000000000000000008_00000_0000000001_00002?
2025-11-27T14:52:23.616388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.616392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000008_00000_0000000001_00002?, parameters.CurOffset=8
2025-11-27T14:52:23.616220Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.616554Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 8.0/3
2025-11-27T14:52:23.616574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 0
2025-11-27T14:52:23.616579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 8:0 appended
2025-11-27T14:52:23.616747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 8.1/3
2025-11-27T14:52:23.616760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 1
2025-11-27T14:52:23.616762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 8:1 appended
2025-11-27T14:52:23.616780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 8.2/3
2025-11-27T14:52:23.616789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 2
2025-11-27T14:52:23.617024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 8 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 1048790 count 1 nextOffset 9 batches 3
2025-11-27T14:52:23.617041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 8:2 appended
2025-11-27T14:52:23.617047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/8] d0000000000_00000000000000000009_00000_0000000001_00002?
2025-11-27T14:52:23.617048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.617051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000009_00000_0000000001_00002?, parameters.CurOffset=9
2025-11-27T14:52:23.617198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 9.0/3
2025-11-27T14:52:23.617214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 0
2025-11-27T14:52:23.617217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 9:0 appended
2025-11-27T14:52:23.617368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 9.1/3
2025-11-27T14:52:23.617381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 1
2025-11-27T14:52:23.617383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 9:1 appended
2025-11-27T14:52:23.617401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 9.2/3
2025-11-27T14:52:23.617404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 2
2025-11-27T14:52:23.617600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 9 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 2097545 count 2 nextOffset 10 batches 5
CallPersQueueGRPC response:
2025-11-27T14:52:23.617617Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 9:2 appended
2025-11-27T14:52:23.617622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/8] d0000000000_00000000000000000010_00000_0000000001_00002?
2025-11-27T14:52:23.617625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.617628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000010_00000_0000000001_00002?, parameters.CurOffset=10
2025-11-27T14:52:23.617630Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.617637Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.617641Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.617645Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.617649Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.617653Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.617657Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.617660Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 2 suffix '63'
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 15
    Offset: 15
    PartNo: 0
    WriteTimestampMS: 1764255143611
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.617774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 10.0/3
2025-11-27T14:52:23.617789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 0
2025-11-27T14:52:23.617792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 10:0 appended
2025-11-27T14:52:23.617938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 10.1/3
2025-11-27T14:52:23.617954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 1
2025-11-27T14:52:23.617959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 10:1 appended
2025-11-27T14:52:23.617972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 10.2/3
2025-11-27T14:52:23.617975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 2
2025-11-27T14:52:23.618176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 10 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 3146300 count 3 nextOffset 11 batches 7
2025-11-27T14:52:23.618193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 10:2 appended
2025-11-27T14:52:23.618197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/8] d0000000000_00000000000000000011_00000_0000000001_00002?
2025-11-27T14:52:23.618199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.618202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000011_00000_0000000001_00002?, parameters.CurOffset=11
2025-11-27T14:52:23.618527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 11.0/3
2025-11-27T14:52:23.618545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 0
2025-11-27T14:52:23.618549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 11:0 appended
2025-11-27T14:52:23.618530Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.618692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 11.1/3
2025-11-27T14:52:23.618705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 1
2025-11-27T14:52:23.618708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 11:1 appended
2025-11-27T14:52:23.618721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 11.2/3
2025-11-27T14:52:23.618724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 2
2025-11-27T14:52:23.618900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 11 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 4195055 count 4 nextOffset 12 batches 9
2025-11-27T14:52:23.618914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 11:2 appended
2025-11-27T14:52:23.618918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/8] d0000000000_00000000000000000012_00000_0000000001_00002?
2025-11-27T14:52:23.618920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.618924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000012_00000_0000000001_00002?, parameters.CurOffset=12
2025-11-27T14:52:23.619068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 12.0/3
2025-11-27T14:52:23.619081Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 0
2025-11-27T14:52:23.619083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 12:0 appended
2025-11-27T14:52:23.619588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 12.1/3
2025-11-27T14:52:23.619601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 1
2025-11-27T14:52:23.619605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 12:1 appended
2025-11-27T14:52:23.619619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 12.2/3
2025-11-27T14:52:23.619622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 2
2025-11-27T14:52:23.619800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 12 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 5243810 count 5 nextOffset 13 batches 11
2025-11-27T14:52:23.619816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 12:2 appended
2025-11-27T14:52:23.619820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/8] d0000000000_00000000000000000013_00000_0000000001_00002?
2025-11-27T14:52:23.619822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.619825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000013_00000_0000000001_00002?, parameters.CurOffset=13
2025-11-27T14:52:23.619971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 13.0/3
2025-11-27T14:52:23.619984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 0
2025-11-27T14:52:23.619987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 13:0 appended
2025-11-27T14:52:23.620486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 13.1/3
2025-11-27T14:52:23.620501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 1
2025-11-27T14:52:23.620504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 13:1 appended
2025-11-27T14:52:23.620518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 13.2/3
2025-11-27T14:52:23.620522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 2
2025-11-27T14:52:23.620697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 13 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 6292565 count 6 nextOffset 14 batches 13
2025-11-27T14:52:23.620712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 13:2 appended
2025-11-27T14:52:23.620716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/8] d0000000000_00000000000000000014_00000_0000000001_00002?
2025-11-27T14:52:23.620718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.620721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000014_00000_0000000001_00002?, parameters.CurOffset=14
2025-11-27T14:52:23.620859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 14.0/3
2025-11-27T14:52:23.620870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 0
2025-11-27T14:52:23.620873Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 14:0 appended
2025-11-27T14:52:23.621004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 14.1/3
2025-11-27T14:52:23.621017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 1
2025-11-27T14:52:23.621020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 14:1 appended
2025-11-27T14:52:23.621032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 14.2/3
2025-11-27T14:52:23.621035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 2
2025-11-27T14:52:23.621384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 14 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 7341320 count 7 nextOffset 15 batches 15
2025-11-27T14:52:23.621401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 14:2 appended
2025-11-27T14:52:23.621405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/8] d0000000000_00000000000000000015_00000_0000000001_00002?
2025-11-27T14:52:23.621406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.621410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000015_00000_0000000001_00002?, parameters.CurOffset=15
2025-11-27T14:52:23.621549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 15.0/3
2025-11-27T14:52:23.621563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 0
2025-11-27T14:52:23.621568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 15:0 appended
2025-11-27T14:52:23.622039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 15.1/3
2025-11-27T14:52:23.622054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 1
2025-11-27T14:52:23.624793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 15 partNo 1 result is x0000000000_00000000000000000007_00002_0000000008_00015 size 7878063
2025-11-27T14:52:23.624814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 15:1 appended
2025-11-27T14:52:23.624832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 15.2/3
2025-11-27T14:52:23.624836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 2
2025-11-27T14:52:23.624847Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1104: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000007_00002_0000000008_00015 new key d0000000000_00000000000000000007_00002_0000000008_00015 size 7878063 WTime 1764255143624
2025-11-27T14:52:23.624976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 15 partNo 2 FormedBlobsCount 1 NewHead: Offset 15 PartNo 1 PackedSize 536763 count 1 nextOffset 16 batches 2
2025-11-27T14:52:23.624994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 15:2 appended
2025-11-27T14:52:23.625047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 15,1 HeadOffset 7 endOffset 8 curOffset 16 d0000000000_00000000000000000015_00001_0000000001_00001| size 536753 WTime 1764255143624
2025-11-27T14:52:23.625170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473869:3423] destroyed
2025-11-27T14:52:23.625186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473874:3426], now have 1 active actors on pipe
2025-11-27T14:52:23.625211Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.625224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.625238Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.625254Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 7 partNo 2 count 8 size 7878063
2025-11-27T14:52:23.625269Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 15 partNo 1 count 1 size 536753
2025-11-27T14:52:23.625304Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f54c5e2f-74cfee28-5db8949d-8659d3_16 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.625319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.625324Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.625329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.625332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.625336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.625348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.625357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.625361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.625374Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.625422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.626006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473874:3426] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f54c5e2f-74cfee28-5db8949d-8659d3_16"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16480
2025-11-27T14:52:23.625581Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f54c5e2f-74cfee28-5db8949d-8659d3_16"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 16
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.633989Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.634623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473881:3430], now have 1 active actors on pipe
2025-11-27T14:52:23.636090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.636113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.636200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 16 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.636271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 16 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.636284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 16 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.636288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 16 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.636310Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.636329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.636333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.636343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.636346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.636350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.636353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.636358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.636386Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 0
2025-11-27T14:52:23.636399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.636402Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 1
2025-11-27T14:52:23.636406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.636409Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 2
2025-11-27T14:52:23.636634Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 16 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 1048790 count 1 nextOffset 17 batches 3
2025-11-27T14:52:23.636646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.636649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.636652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.636726Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 16,1 HeadOffset 16 endOffset 16 curOffset 17 d0000000000_00000000000000000016_00000_0000000001_00002? size 1048780 WTime 1764255143636
2025-11-27T14:52:23.637037Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.637050Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 16 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.644198Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 16
    Offset: 16
    PartNo: 0
    WriteTimestampMS: 1764255143636
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 7
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.642900Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 7 count 8 size 7878063 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.642918Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 15 count 1 size 536753 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.642938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.642990Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 7 partno 2 count 8 parts 15 suffix '0' size 7878063
2025-11-27T14:52:23.642997Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 15 partno 1 count 1 parts 1 suffix '124' size 536753
2025-11-27T14:52:23.643093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-27T14:52:23.643099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.643906Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 16 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.643923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.643933Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.643951Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.643958Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.643968Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 16, partNo: 0, Offset: 16 is stored on disk
2025-11-27T14:52:23.643973Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.643977Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 16, partNo: 1, Offset: 16 is stored on disk
2025-11-27T14:52:23.643982Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.643987Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 16, partNo: 2, Offset: 16 is stored on disk
2025-11-27T14:52:23.644019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.644022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.644025Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.644027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.644030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.644074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.644091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.644107Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.644115Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000008_00000_0000000001_00002?(+) to d0000000000_00000000000000000008_00000_0000000001_00002?(+)
2025-11-27T14:52:23.644117Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000009_00000_0000000001_00002?(+) to d0000000000_00000000000000000009_00000_0000000001_00002?(+)
2025-11-27T14:52:23.644120Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000010_00000_0000000001_00002?(+) to d0000000000_00000000000000000010_00000_0000000001_00002?(+)
2025-11-27T14:52:23.644122Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000011_00000_0000000001_00002?(+) to d0000000000_00000000000000000011_00000_0000000001_00002?(+)
2025-11-27T14:52:23.644127Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000012_00000_0000000001_00002?(+) to d0000000000_00000000000000000012_00000_0000000001_00002?(+)
2025-11-27T14:52:23.644129Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000013_00000_0000000001_00002?(+) to d0000000000_00000000000000000013_00000_0000000001_00002?(+)
2025-11-27T14:52:23.644132Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000014_00000_0000000001_00002?(+) to d0000000000_00000000000000000014_00000_0000000001_00002?(+)
2025-11-27T14:52:23.644134Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000015_00000_0000000001_00002?(+) to d0000000000_00000000000000000015_00000_0000000001_00002?(+)
2025-11-27T14:52:23.644150Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000007_00002_0000000001_00000|(+) to d0000000000_00000000000000000007_00002_0000000001_00000|(+)
2025-11-27T14:52:23.644643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473881:3430] destroyed
2025-11-27T14:52:23.644960Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 8 count 1 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.644966Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 9 count 1 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.644971Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 10 count 1 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.644975Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 11 count 1 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.644982Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 12 count 1 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.644986Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 13 count 1 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.644991Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 14 count 1 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.644995Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 15 count 1 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.645001Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 7 count 1 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.645016Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.645017Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.645027Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.645028Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.645034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.645036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.645037Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.645038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.645041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.645042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.645045Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.645049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.645051Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.645059Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.645064Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.645069Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.645075Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 7 partno 2 count 1 parts 0 suffix '124' size 24726
2025-11-27T14:52:23.647082Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.647631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473889:3436], now have 1 active actors on pipe
2025-11-27T14:52:23.647967Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.647780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.647797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.647850Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|3ea8151e-c8f518c-3a115f01-308141a7_17 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.647859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.647863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
CallPersQueueGRPC response:
2025-11-27T14:52:23.647867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.647870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.647874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.647884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.647887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.647891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.647906Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.647919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.648218Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473889:3436] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|3ea8151e-c8f518c-3a115f01-308141a7_17"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|3ea8151e-c8f518c-3a115f01-308141a7_17"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 17
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.658125Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.658632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473893:3438], now have 1 active actors on pipe
2025-11-27T14:52:23.659932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.659953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.660027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 17 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.660093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 17 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.660110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 17 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.660116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 17 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.660135Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.660152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.660157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.660167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.660170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.660174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.660177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.660185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.660215Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 0
2025-11-27T14:52:23.660230Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.660233Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 1
2025-11-27T14:52:23.660237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.660241Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 2
2025-11-27T14:52:23.660662Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 17 partNo 2 FormedBlobsCount 0 NewHead: Offset 17 PartNo 0 PackedSize 1048790 count 1 nextOffset 18 batches 3
2025-11-27T14:52:23.660686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.660690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.660693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.660772Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 17,1 HeadOffset 17 endOffset 17 curOffset 18 d0000000000_00000000000000000017_00000_0000000001_00002? size 1048780 WTime 1764255143660
2025-11-27T14:52:23.660945Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.660969Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 17 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.663953Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 17 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.663981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.664003Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.664004Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.664010Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.664020Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 17, partNo: 0, Offset: 17 is stored on disk
2025-11-27T14:52:23.664024Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.664031Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 17, partNo: 1, Offset: 17 is stored on disk
2025-11-27T14:52:23.664037Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.664043Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 17, partNo: 2, Offset: 17 is stored on disk
2025-11-27T14:52:23.664088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.664097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.664100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.664102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.664104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.664114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.664130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.664197Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.664550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473893:3438] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 17
    Offset: 17
    PartNo: 0
    WriteTimestampMS: 1764255143659
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.665450Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.665939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473898:3441], now have 1 active actors on pipe
2025-11-27T14:52:23.666071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.666085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.666118Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|68d00eb2-d6e3f09e-6b8fa8df-9d208563_18 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.666135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.666138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.666142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.666145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.666148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.666157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.666161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.666164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.666178Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.666190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.666257Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.666485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473898:3441] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|68d00eb2-d6e3f09e-6b8fa8df-9d208563_18"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|68d00eb2-d6e3f09e-6b8fa8df-9d208563_18"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 18
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.673251Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.673691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473901:3442], now have 1 active actors on pipe
2025-11-27T14:52:23.674983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.675006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.675126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 18 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.675188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 18 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.675204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 18 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.675209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 18 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.675230Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.675247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.675253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.675266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.675272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.675274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.675277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.675281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.675312Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 0
2025-11-27T14:52:23.675330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.675336Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 1
2025-11-27T14:52:23.675338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.675340Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 2
2025-11-27T14:52:23.675585Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 18 partNo 2 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 1048790 count 1 nextOffset 19 batches 3
2025-11-27T14:52:23.675605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.675608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.675611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.675692Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 18,1 HeadOffset 18 endOffset 18 curOffset 19 d0000000000_00000000000000000018_00000_0000000001_00002? size 1048780 WTime 1764255143675
2025-11-27T14:52:23.675955Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.675977Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 18 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.678843Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 18 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.678869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.678888Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.678897Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.678904Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.678913Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 18, partNo: 0, Offset: 18 is stored on disk
2025-11-27T14:52:23.678924Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.678929Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 18, partNo: 1, Offset: 18 is stored on disk
2025-11-27T14:52:23.678935Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.678946Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 18, partNo: 2, Offset: 18 is stored on disk
2025-11-27T14:52:23.678976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.678984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.678986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.678988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.678990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.678995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.679005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.679095Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 18
    Offset: 18
    PartNo: 0
    WriteTimestampMS: 1764255143675
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16480
2025-11-27T14:52:23.679626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473901:3442] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.680264Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.680681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473906:3445], now have 1 active actors on pipe
2025-11-27T14:52:23.680816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.680837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.680877Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|fb547a5d-24ed325f-f1976473-1731e4d1_19 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.680894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.680898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.680905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.680913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.680917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.680929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.680932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.680936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.680953Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.680977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.681052Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|fb547a5d-24ed325f-f1976473-1731e4d1_19"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16480
2025-11-27T14:52:23.681451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473906:3445] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|fb547a5d-24ed325f-f1976473-1731e4d1_19"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 19
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.687845Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.687965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.687985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.687988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.687992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.687995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.688443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473909:3446], now have 1 active actors on pipe
2025-11-27T14:52:23.689849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.689871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.689962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 19 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.690026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 19 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.690047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 19 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.690053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 19 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.690077Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.690097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.690105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.690115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.690118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.690121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.690124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.690129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.690166Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 0
2025-11-27T14:52:23.690180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.690185Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 1
2025-11-27T14:52:23.690188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.690192Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 2
2025-11-27T14:52:23.690424Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 19 partNo 2 FormedBlobsCount 0 NewHead: Offset 19 PartNo 0 PackedSize 1048790 count 1 nextOffset 20 batches 3
2025-11-27T14:52:23.690435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.690439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.690442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.690525Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 19,1 HeadOffset 19 endOffset 19 curOffset 20 d0000000000_00000000000000000019_00000_0000000001_00002? size 1048780 WTime 1764255143690
2025-11-27T14:52:23.690749Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.690781Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 19 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.703954Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 19 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.704001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.704031Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.704039Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.704050Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 19, partNo: 0, Offset: 19 is stored on disk
2025-11-27T14:52:23.704056Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.704075Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 19, partNo: 1, Offset: 19 is stored on disk
2025-11-27T14:52:23.704081Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.704088Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 19, partNo: 2, Offset: 19 is stored on disk
2025-11-27T14:52:23.704129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.704132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.704135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.704137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.704140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.704150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.704166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.707485Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.707329Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 2 suffix '63' size 1048780
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 19
    Offset: 19
    PartNo: 0
    WriteTimestampMS: 1764255143689
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 13
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.707967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473909:3446] destroyed
2025-11-27T14:52:23.710631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473914:3449], now have 1 active actors on pipe
2025-11-27T14:52:23.710763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.710780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.710824Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|13feff54-e2d1c51d-75e4452e-c0fc3eb6_20 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.710835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.710840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.710845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.710848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.710853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.710864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.710867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.710870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.710889Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.710907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.710052Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.711856Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.712206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473914:3449] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|13feff54-e2d1c51d-75e4452e-c0fc3eb6_20"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|13feff54-e2d1c51d-75e4452e-c0fc3eb6_20"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 20
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.719233Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.719948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473917:3450], now have 1 active actors on pipe
2025-11-27T14:52:23.721207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.721228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.721346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 20 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.721410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 20 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.721423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 20 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.721430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 20 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.721451Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.721473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.721480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.721489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.721491Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.721494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.721495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.721503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.721532Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 0
2025-11-27T14:52:23.721551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.721554Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 1
2025-11-27T14:52:23.721557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.721559Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 2
2025-11-27T14:52:23.721778Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 20 partNo 2 FormedBlobsCount 0 NewHead: Offset 20 PartNo 0 PackedSize 1048790 count 1 nextOffset 21 batches 3
2025-11-27T14:52:23.721789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.721793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.721796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.721865Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 20,1 HeadOffset 20 endOffset 20 curOffset 21 d0000000000_00000000000000000020_00000_0000000001_00002? size 1048780 WTime 1764255143721
2025-11-27T14:52:23.722197Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.722207Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 20 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.725514Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 20 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.725551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.725578Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.725582Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.725585Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.725597Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 20, partNo: 0, Offset: 20 is stored on disk
2025-11-27T14:52:23.725602Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.725607Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 20, partNo: 1, Offset: 20 is stored on disk
2025-11-27T14:52:23.725612Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.725621Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 20, partNo: 2, Offset: 20 is stored on disk
2025-11-27T14:52:23.725662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.725666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.725668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.725671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.725676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.725684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.725700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 20
    Offset: 20
    PartNo: 0
    WriteTimestampMS: 1764255143721
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-27T14:52:23.725796Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.726280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473917:3450] destroyed
2025-11-27T14:52:23.727847Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.728480Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473922:3453], now have 1 active actors on pipe
2025-11-27T14:52:23.728631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.728642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.728682Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a77e01f3-66cf0bc4-b157d2ac-d0ee5009_21 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.728693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.728697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.728703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.728705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.728709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.728720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.728723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.728726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.728743Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.728758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.728819Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a77e01f3-66cf0bc4-b157d2ac-d0ee5009_21"
    Status: Active
    SeqNo: 0
  }
}
2025-11-27T14:52:23.729280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473922:3453] destroyed
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a77e01f3-66cf0bc4-b157d2ac-d0ee5009_21"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 21
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.736706Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.737194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473925:3454], now have 1 active actors on pipe
2025-11-27T14:52:23.738537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.738560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.738665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 21 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.738738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 21 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.738759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 21 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.738764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 21 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.738789Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.738813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.738817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.738828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.738830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.738835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.738838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.738843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.738882Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 0
2025-11-27T14:52:23.738900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.738903Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 1
2025-11-27T14:52:23.738908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.738914Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 2
2025-11-27T14:52:23.739157Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 21 partNo 2 FormedBlobsCount 0 NewHead: Offset 21 PartNo 0 PackedSize 1048790 count 1 nextOffset 22 batches 3
2025-11-27T14:52:23.739168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.739172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.739175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.739254Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 21,1 HeadOffset 21 endOffset 21 curOffset 22 d0000000000_00000000000000000021_00000_0000000001_00002? size 1048780 WTime 1764255143739
2025-11-27T14:52:23.739682Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.739709Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 21 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.745398Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 21 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.745435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.745463Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.745474Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.745478Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.745489Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 21, partNo: 0, Offset: 21 is stored on disk
2025-11-27T14:52:23.745494Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.745500Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 21, partNo: 1, Offset: 21 is stored on disk
2025-11-27T14:52:23.745506Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.745511Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 21, partNo: 2, Offset: 21 is stored on disk
2025-11-27T14:52:23.745550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.745560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.745563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.745565Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.745568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.745577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.745593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.745701Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.746235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473925:3454] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 21
    Offset: 21
    PartNo: 0
    WriteTimestampMS: 1764255143738
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 6
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.752578Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.753176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473930:3457], now have 1 active actors on pipe
2025-11-27T14:52:23.753342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.753357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.753399Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|85cfaba2-e5d7d520-cf6e9a46-453bba67_22 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.753416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.753420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.753427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.753430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.753435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.753447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.753450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.753453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.753471Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.753493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.753618Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.753947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473930:3457] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|85cfaba2-e5d7d520-cf6e9a46-453bba67_22"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|85cfaba2-e5d7d520-cf6e9a46-453bba67_22"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 22
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.759276Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.760097Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473933:3458], now have 1 active actors on pipe
2025-11-27T14:52:23.761420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.761458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.761544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 22 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.761602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 22 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.761614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 22 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.761621Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 22 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.761644Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.761662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.761665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.761676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.761680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.761684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.761686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.761692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.761724Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 0
2025-11-27T14:52:23.761739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.761742Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 1
2025-11-27T14:52:23.761745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.761749Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 2
2025-11-27T14:52:23.761971Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 22 partNo 2 FormedBlobsCount 0 NewHead: Offset 22 PartNo 0 PackedSize 1048790 count 1 nextOffset 23 batches 3
2025-11-27T14:52:23.761982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.761988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.761993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.762069Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 22,1 HeadOffset 22 endOffset 22 curOffset 23 d0000000000_00000000000000000022_00000_0000000001_00002? size 1048780 WTime 1764255143761
2025-11-27T14:52:23.762339Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.762353Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 22 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.766201Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 22 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.766222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.766249Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.766256Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.766266Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 22, partNo: 0, Offset: 22 is stored on disk
2025-11-27T14:52:23.766271Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.766276Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 22, partNo: 1, Offset: 22 is stored on disk
2025-11-27T14:52:23.766281Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.766288Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 22, partNo: 2, Offset: 22 is stored on disk
2025-11-27T14:52:23.766324Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.766327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.766330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.766333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.766335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.766345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.766360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.766489Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.766599Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 22
    Offset: 22
    PartNo: 0
    WriteTimestampMS: 1764255143761
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.767202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473933:3458] destroyed
2025-11-27T14:52:23.767790Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.768317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473938:3461], now have 1 active actors on pipe
2025-11-27T14:52:23.768354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.768362Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.768406Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|18649c2a-a80b4f1f-1b436345-b2043beb_23 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.768424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.768428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.768436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.768441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.768446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.768458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.768461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.768465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.768482Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.768499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.768589Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|18649c2a-a80b4f1f-1b436345-b2043beb_23"
    Status: Active
    SeqNo: 0
  }
}
2025-11-27T14:52:23.768895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473938:3461] destroyed
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|18649c2a-a80b4f1f-1b436345-b2043beb_23"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 23
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.775188Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.775734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473941:3462], now have 1 active actors on pipe
2025-11-27T14:52:23.777048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.777072Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.777161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 23 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.777217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 23 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.777227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 23 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.777232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 23 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.777259Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.777279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.777284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.777294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.777297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.777302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.777304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.777310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.777346Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 0
2025-11-27T14:52:23.777361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.777365Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 1
2025-11-27T14:52:23.777369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.777373Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 2
2025-11-27T14:52:23.777606Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 23 partNo 2 FormedBlobsCount 0 NewHead: Offset 23 PartNo 0 PackedSize 1048790 count 1 nextOffset 24 batches 3
2025-11-27T14:52:23.777616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.777622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.777626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.777704Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 23,1 HeadOffset 23 endOffset 23 curOffset 24 d0000000000_00000000000000000023_00000_0000000001_00002? size 1048780 WTime 1764255143777
2025-11-27T14:52:23.777886Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.777899Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 23 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.781640Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 23 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.781670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.781691Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.781695Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.781701Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.781707Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 23, partNo: 0, Offset: 23 is stored on disk
2025-11-27T14:52:23.781711Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.781714Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 23, partNo: 1, Offset: 23 is stored on disk
2025-11-27T14:52:23.781719Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.781722Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 23, partNo: 2, Offset: 23 is stored on disk
2025-11-27T14:52:23.781750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.781756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.781757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.781759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.781763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.781771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000016_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.781773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000017_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.781775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000018_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.781777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000019_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.781779Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.781781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.781782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.781784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000023_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.781786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 8 keys were taken away. Let's read 8390240 bytes
2025-11-27T14:52:23.781798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.781809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 8 blobs
2025-11-27T14:52:23.781811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000016_00000_0000000001_00002?
2025-11-27T14:52:23.781820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000017_00000_0000000001_00002?
2025-11-27T14:52:23.781826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000018_00000_0000000001_00002?
2025-11-27T14:52:23.781833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000019_00000_0000000001_00002?
2025-11-27T14:52:23.781838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00000_0000000001_00002?
2025-11-27T14:52:23.781848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00002?
2025-11-27T14:52:23.781854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000001_00002?
2025-11-27T14:52:23.781860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000023_00000_0000000001_00002?
2025-11-27T14:52:23.781869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 8 blobs for compaction
2025-11-27T14:52:23.781886Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 16 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.781892Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 17 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.781895Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 18 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.781897Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 19 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.781899Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 20 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.781901Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 21 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.781903Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 22 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.781904Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 23 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.781919Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 8 blobs are from cache.
2025-11-27T14:52:23.781947Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.781950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-27T14:52:23.781956Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.781959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/8] d0000000000_00000000000000000016_00000_0000000001_00002?
2025-11-27T14:52:23.781960Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.781962Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.781964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000016_00000_0000000001_00002?, parameters.CurOffset=16
2025-11-27T14:52:23.781976Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.781979Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.781983Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.781988Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.781994Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.782106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 16.0/3
2025-11-27T14:52:23.782120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 0
2025-11-27T14:52:23.782125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 16:0 appended
2025-11-27T14:52:23.782263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 16.1/3
2025-11-27T14:52:23.782272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 1
2025-11-27T14:52:23.782275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 16:1 appended
2025-11-27T14:52:23.782291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 16.2/3
2025-11-27T14:52:23.782294Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 2
2025-11-27T14:52:23.782519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 16 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 1048790 count 1 nextOffset 17 batches 3
2025-11-27T14:52:23.782532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 16:2 appended
2025-11-27T14:52:23.782535Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/8] d0000000000_00000000000000000017_00000_0000000001_00002?
2025-11-27T14:52:23.782536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.782538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000017_00000_0000000001_00002?, parameters.CurOffset=17
2025-11-27T14:52:23.782673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 17.0/3
2025-11-27T14:52:23.782683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 0
2025-11-27T14:52:23.782686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 17:0 appended
2025-11-27T14:52:23.782824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 17.1/3
2025-11-27T14:52:23.782832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 1
2025-11-27T14:52:23.782834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 17:1 appended
2025-11-27T14:52:23.782846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 17.2/3
2025-11-27T14:52:23.782848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 2
CallPersQueueGRPC response:
2025-11-27T14:52:23.781883Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 23
    Offset: 23
    PartNo: 0
    WriteTimestampMS: 1764255143776
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.783031Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 17 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 2097545 count 2 nextOffset 18 batches 5
2025-11-27T14:52:23.783042Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 17:2 appended
2025-11-27T14:52:23.783044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/8] d0000000000_00000000000000000018_00000_0000000001_00002?
2025-11-27T14:52:23.783045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.783047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000018_00000_0000000001_00002?, parameters.CurOffset=18
2025-11-27T14:52:23.783534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 18.0/3
2025-11-27T14:52:23.783546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 0
2025-11-27T14:52:23.783549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 18:0 appended
2025-11-27T14:52:23.783692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 18.1/3
2025-11-27T14:52:23.783699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 1
2025-11-27T14:52:23.783701Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 18:1 appended
2025-11-27T14:52:23.783711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 18.2/3
2025-11-27T14:52:23.783713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 2
2025-11-27T14:52:23.783662Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.783896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 18 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 3146300 count 3 nextOffset 19 batches 7
2025-11-27T14:52:23.783906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 18:2 appended
2025-11-27T14:52:23.783909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/8] d0000000000_00000000000000000019_00000_0000000001_00002?
2025-11-27T14:52:23.783912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.783914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000019_00000_0000000001_00002?, parameters.CurOffset=19
2025-11-27T14:52:23.784085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 19.0/3
2025-11-27T14:52:23.784098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 0
2025-11-27T14:52:23.784102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 19:0 appended
2025-11-27T14:52:23.784260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 19.1/3
2025-11-27T14:52:23.784271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 1
2025-11-27T14:52:23.784274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 19:1 appended
2025-11-27T14:52:23.784287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 19.2/3
2025-11-27T14:52:23.784290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 2
2025-11-27T14:52:23.784473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 19 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 4195055 count 4 nextOffset 20 batches 9
2025-11-27T14:52:23.784483Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 19:2 appended
2025-11-27T14:52:23.784485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/8] d0000000000_00000000000000000020_00000_0000000001_00002?
2025-11-27T14:52:23.784487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.784489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000020_00000_0000000001_00002?, parameters.CurOffset=20
2025-11-27T14:52:23.784968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 20.0/3
2025-11-27T14:52:23.784981Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 0
2025-11-27T14:52:23.784983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 20:0 appended
2025-11-27T14:52:23.785488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 20.1/3
2025-11-27T14:52:23.785503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 1
2025-11-27T14:52:23.785507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 20:1 appended
2025-11-27T14:52:23.785520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 20.2/3
2025-11-27T14:52:23.785523Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 2
2025-11-27T14:52:23.785698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 20 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 5243810 count 5 nextOffset 21 batches 11
2025-11-27T14:52:23.785709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 20:2 appended
2025-11-27T14:52:23.785712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/8] d0000000000_00000000000000000021_00000_0000000001_00002?
2025-11-27T14:52:23.785713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.785714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000021_00000_0000000001_00002?, parameters.CurOffset=21
2025-11-27T14:52:23.785854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 21.0/3
2025-11-27T14:52:23.785863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 0
2025-11-27T14:52:23.785865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 21:0 appended
2025-11-27T14:52:23.786001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 21.1/3
2025-11-27T14:52:23.786009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 1
2025-11-27T14:52:23.786010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 21:1 appended
2025-11-27T14:52:23.786019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 21.2/3
2025-11-27T14:52:23.786021Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 2
2025-11-27T14:52:23.786238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 21 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 6292565 count 6 nextOffset 22 batches 13
2025-11-27T14:52:23.786248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 21:2 appended
2025-11-27T14:52:23.786250Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/8] d0000000000_00000000000000000022_00000_0000000001_00002?
2025-11-27T14:52:23.786252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.786253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000022_00000_0000000001_00002?, parameters.CurOffset=22
2025-11-27T14:52:23.786400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 22.0/3
2025-11-27T14:52:23.786409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 0
2025-11-27T14:52:23.786410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 22:0 appended
2025-11-27T14:52:23.786937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 22.1/3
2025-11-27T14:52:23.786949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 1
2025-11-27T14:52:23.786951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 22:1 appended
2025-11-27T14:52:23.786962Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 22.2/3
2025-11-27T14:52:23.786964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 2
2025-11-27T14:52:23.787805Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 22 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 7341320 count 7 nextOffset 23 batches 15
2025-11-27T14:52:23.787817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 22:2 appended
2025-11-27T14:52:23.787821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/8] d0000000000_00000000000000000023_00000_0000000001_00002?
2025-11-27T14:52:23.787822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.787824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000023_00000_0000000001_00002?, parameters.CurOffset=23
2025-11-27T14:52:23.787955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 23.0/3
2025-11-27T14:52:23.787965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 0
2025-11-27T14:52:23.793859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 23 partNo 0 result is x0000000000_00000000000000000015_00001_0000000008_00015 size 7878063
2025-11-27T14:52:23.793883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 23:0 appended
2025-11-27T14:52:23.794023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 23.1/3
2025-11-27T14:52:23.794039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 1
2025-11-27T14:52:23.794042Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 23:1 appended
2025-11-27T14:52:23.794061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 23.2/3
2025-11-27T14:52:23.794064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 2
2025-11-27T14:52:23.794078Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1104: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000015_00001_0000000008_00015 new key d0000000000_00000000000000000015_00001_0000000008_00015 size 7878063 WTime 1764255143793
2025-11-27T14:52:23.794316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 23 partNo 2 FormedBlobsCount 1 NewHead: Offset 23 PartNo 0 PackedSize 1048790 count 1 nextOffset 24 batches 3
2025-11-27T14:52:23.794335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 23:2 appended
2025-11-27T14:52:23.794389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 23,1 HeadOffset 15 endOffset 16 curOffset 24 d0000000000_00000000000000000023_00000_0000000001_00002| size 1048780 WTime 1764255143793
2025-11-27T14:52:23.794543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473941:3462] destroyed
2025-11-27T14:52:23.794555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473946:3465], now have 1 active actors on pipe
2025-11-27T14:52:23.794569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.794577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.794593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.794599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.794600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.794602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.794603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.794618Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.794630Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 15 partNo 1 count 8 size 7878063
2025-11-27T14:52:23.794639Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 23 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.794667Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|6d5f987a-9cd002d3-80735d64-7cfd1c97_24 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.794678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.794682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.794685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.794687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.794691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.794700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.794702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.794704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.794715Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.794769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.795197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473946:3465] destroyed
2025-11-27T14:52:23.794848Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|6d5f987a-9cd002d3-80735d64-7cfd1c97_24"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|6d5f987a-9cd002d3-80735d64-7cfd1c97_24"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 24
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.803416Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.803967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473952:3469], now have 1 active actors on pipe
2025-11-27T14:52:23.805356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.805382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.805481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 24 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.805549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 24 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.805579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 24 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.805587Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 24 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.805611Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.805634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.805645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.805654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.805662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.805670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.805672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.805677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.805708Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 0
2025-11-27T14:52:23.805728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.805737Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 1
2025-11-27T14:52:23.805740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.805743Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 2
2025-11-27T14:52:23.805968Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 24 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 1048790 count 1 nextOffset 25 batches 3
2025-11-27T14:52:23.805990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.805993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.805997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.806069Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 24,1 HeadOffset 24 endOffset 24 curOffset 25 d0000000000_00000000000000000024_00000_0000000001_00002? size 1048780 WTime 1764255143805
2025-11-27T14:52:23.806264Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.806290Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 24 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.813331Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 15 count 8 size 7878063 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.813354Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 23 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.813393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.813399Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 15 partno 1 count 8 parts 15 suffix '0' size 7878063
2025-11-27T14:52:23.813409Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 7 parts 16 suffix '0' size 8365339
2025-11-27T14:52:23.813424Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 2 suffix '124' size 1048780
2025-11-27T14:52:23.813608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-27T14:52:23.813625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.813810Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 0 size 8365339 cause it's been evicted from L2. Actual L1 size: 12
2025-11-27T14:52:23.813823Z node 2 :PERSQUEUE NOTICE: read.h:372: [72075186224037892][PQCacheProxy]Have to remove new data from cache. Topic rt3.dc1--topic1, tablet id72075186224037892, cookie 0
2025-11-27T14:52:23.814839Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 24 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.814870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.814900Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.814906Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.814911Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.814922Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 24, partNo: 0, Offset: 24 is stored on disk
2025-11-27T14:52:23.814935Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.814939Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 24, partNo: 1, Offset: 24 is stored on disk
2025-11-27T14:52:23.814945Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.814950Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 24, partNo: 2, Offset: 24 is stored on disk
2025-11-27T14:52:23.814987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.814996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.814998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.815001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.815003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.815036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.815060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.815080Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.815093Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000016_00000_0000000001_00002?(+) to d0000000000_00000000000000000016_00000_0000000001_00002?(+)
2025-11-27T14:52:23.815096Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000017_00000_0000000001_00002?(+) to d0000000000_00000000000000000017_00000_0000000001_00002?(+)
2025-11-27T14:52:23.815098Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000018_00000_0000000001_00002?(+) to d0000000000_00000000000000000018_00000_0000000001_00002?(+)
2025-11-27T14:52:23.815101Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000019_00000_0000000001_00002?(+) to d0000000000_00000000000000000019_00000_0000000001_00002?(+)
2025-11-27T14:52:23.815104Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000020_00000_0000000001_00002?(+) to d0000000000_00000000000000000020_00000_0000000001_00002?(+)
2025-11-27T14:52:23.815106Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000021_00000_0000000001_00002?(+) to d0000000000_00000000000000000021_00000_0000000001_00002?(+)
2025-11-27T14:52:23.815108Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000022_00000_0000000001_00002?(+) to d0000000000_00000000000000000022_00000_0000000001_00002?(+)
2025-11-27T14:52:23.815111Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000023_00000_0000000001_00002?(+) to d0000000000_00000000000000000023_00000_0000000001_00002?(+)
2025-11-27T14:52:23.815114Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000015_00001_0000000001_00001|(+) to d0000000000_00000000000000000015_00001_0000000001_00001|(+)
CallPersQueueGRPC response:
2025-11-27T14:52:23.815678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473952:3469] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 24
    Offset: 24
    PartNo: 0
    WriteTimestampMS: 1764255143805
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 9
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.816238Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 16 count 1 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.816249Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 17 count 1 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.816255Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 18 count 1 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.816259Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 19 count 1 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.816265Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 20 count 1 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.816269Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 21 count 1 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.816274Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 22 count 1 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.816278Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 23 count 1 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.816285Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 15 count 1 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.816307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.816323Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.816326Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.816333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.816335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.816337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.816341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.816343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.816348Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.816349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.816355Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.816362Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.816368Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.816376Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.816382Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.816389Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.816394Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 15 partno 1 count 1 parts 1 suffix '124' size 536753
2025-11-27T14:52:23.815176Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.817026Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.817524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473960:3474], now have 1 active actors on pipe
2025-11-27T14:52:23.817669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.817692Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.817710Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|7d4755d7-1a894ead-e39f576b-fa5abf29_25 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.817715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.817718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.817722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.817723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.817727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.817735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.817742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.817745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.817753Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.817761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.817809Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.818071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473960:3474] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|7d4755d7-1a894ead-e39f576b-fa5abf29_25"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|7d4755d7-1a894ead-e39f576b-fa5abf29_25"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 25
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.825159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473964:3477], now have 1 active actors on pipe
2025-11-27T14:52:23.826567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.826596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.826689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 25 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.826766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 25 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.826787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 25 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.826793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 25 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.826820Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.826847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.826857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.826868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.826872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.826879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.826881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.826887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.826924Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 0
2025-11-27T14:52:23.826950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.826955Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 1
2025-11-27T14:52:23.826963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.826973Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 2
2025-11-27T14:52:23.827229Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 25 partNo 2 FormedBlobsCount 0 NewHead: Offset 25 PartNo 0 PackedSize 1048790 count 1 nextOffset 26 batches 3
2025-11-27T14:52:23.827253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.827257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.827261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.827338Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 25,1 HeadOffset 25 endOffset 25 curOffset 26 d0000000000_00000000000000000025_00000_0000000001_00002? size 1048780 WTime 1764255143827
2025-11-27T14:52:23.824490Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.827651Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.827675Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 25 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.832926Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 25 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.832960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.832990Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.832998Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.833012Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 25, partNo: 0, Offset: 25 is stored on disk
2025-11-27T14:52:23.833017Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.833022Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 25, partNo: 1, Offset: 25 is stored on disk
2025-11-27T14:52:23.833030Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.833035Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 25, partNo: 2, Offset: 25 is stored on disk
2025-11-27T14:52:23.833075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.833087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.833090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.833093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.833095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.833106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.833125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.833186Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 2 suffix '63' size 1048780
CallPersQueueGRPC response:
2025-11-27T14:52:23.833681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473964:3477] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 25
    Offset: 25
    PartNo: 0
    WriteTimestampMS: 1764255143826
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 6
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.834811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473969:3480], now have 1 active actors on pipe
2025-11-27T14:52:23.834920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.834935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.834974Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|c29cc094-6de6cef6-7e587770-33aa0b56_26 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.834991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.834994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.834999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.835001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.835007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.835016Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.835019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.835022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.835034Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.835046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
2025-11-27T14:52:23.835398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473969:3480] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|c29cc094-6de6cef6-7e587770-33aa0b56_26"
    Status: Active
    SeqNo: 0
  }
}
2025-11-27T14:52:23.833290Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.834397Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
CallPersQueueGRPC request to localhost:2025-11-27T14:52:23.835126Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|c29cc094-6de6cef6-7e587770-33aa0b56_26"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 26
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.845228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473972:3481], now have 1 active actors on pipe
2025-11-27T14:52:23.847275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.847300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.847385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 26 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.844619Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.847856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 26 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.847878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 26 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.847885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 26 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.847911Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.847939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.847948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.847959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.847969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.847974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.847976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.847982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.848014Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 0
2025-11-27T14:52:23.848034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.848040Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 1
2025-11-27T14:52:23.848044Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.848048Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 2
2025-11-27T14:52:23.849030Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 26 partNo 2 FormedBlobsCount 0 NewHead: Offset 26 PartNo 0 PackedSize 1048790 count 1 nextOffset 27 batches 3
2025-11-27T14:52:23.849055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.849062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.849065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.849140Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 26,1 HeadOffset 26 endOffset 26 curOffset 27 d0000000000_00000000000000000026_00000_0000000001_00002? size 1048780 WTime 1764255143848
2025-11-27T14:52:23.849326Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.849349Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 26 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.852403Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 26 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.852434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.852456Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.852460Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.852465Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.852474Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 26, partNo: 0, Offset: 26 is stored on disk
2025-11-27T14:52:23.852479Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.852486Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 26, partNo: 1, Offset: 26 is stored on disk
2025-11-27T14:52:23.852492Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.852497Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 26, partNo: 2, Offset: 26 is stored on disk
2025-11-27T14:52:23.852530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.852533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.852535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.852538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.852540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.852549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.852564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.852663Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.852952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473972:3481] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 26
    Offset: 26
    PartNo: 0
    WriteTimestampMS: 1764255143847
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.855954Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.856459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473979:3485], now have 1 active actors on pipe
2025-11-27T14:52:23.856556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.856570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.856611Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|4bddc622-631ab87c-649eee12-4b848538_27 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.856627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.856631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.856636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.856640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.856647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.856661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.856668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.856671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.856686Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.856705Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.856782Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.857031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473979:3485] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|4bddc622-631ab87c-649eee12-4b848538_27"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|4bddc622-631ab87c-649eee12-4b848538_27"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 27
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.862525Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.862962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473982:3486], now have 1 active actors on pipe
2025-11-27T14:52:23.864216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.864236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.864388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 27 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.864466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 27 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.864484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 27 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.864489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 27 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.864508Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.864528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.864536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.864547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.864552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.864557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.864559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.864564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.864589Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 0
2025-11-27T14:52:23.864608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.864616Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 1
2025-11-27T14:52:23.864620Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.864623Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 2
2025-11-27T14:52:23.864835Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 27 partNo 2 FormedBlobsCount 0 NewHead: Offset 27 PartNo 0 PackedSize 1048790 count 1 nextOffset 28 batches 3
2025-11-27T14:52:23.864858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.864861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.864864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.864939Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 27,1 HeadOffset 27 endOffset 27 curOffset 28 d0000000000_00000000000000000027_00000_0000000001_00002? size 1048780 WTime 1764255143864
2025-11-27T14:52:23.865119Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.865143Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 27 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.868495Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 27 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.868523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.868534Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.868545Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.868551Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.868563Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 27, partNo: 0, Offset: 27 is stored on disk
2025-11-27T14:52:23.868570Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.868576Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 27, partNo: 1, Offset: 27 is stored on disk
2025-11-27T14:52:23.868625Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.868631Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 27, partNo: 2, Offset: 27 is stored on disk
2025-11-27T14:52:23.868660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.868668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.868670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.868672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.868675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.868683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.868697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.868772Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 27
    Offset: 27
    PartNo: 0
    WriteTimestampMS: 1764255143864
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-27T14:52:23.869068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473982:3486] destroyed
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.869637Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.870022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473988:3489], now have 1 active actors on pipe
2025-11-27T14:52:23.870114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.870125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.870157Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|747f80b3-af732000-c29939cc-234ca1d5_28 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.870165Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.870168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.870173Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.870175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.870178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.870185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.870187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.870190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.870202Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.870212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.870363Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|747f80b3-af732000-c29939cc-234ca1d5_28"
    Status: Active
    SeqNo: 0
  }
}
2025-11-27T14:52:23.870584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473988:3489] destroyed
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|747f80b3-af732000-c29939cc-234ca1d5_28"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 28
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.876578Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.877056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473991:3490], now have 1 active actors on pipe
2025-11-27T14:52:23.878104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.878126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.878259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 28 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.878337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 28 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.878355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 28 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.878365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 28 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.878387Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.878411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.878421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.878433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.878441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.878446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.878452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.878457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.878489Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 0
2025-11-27T14:52:23.878507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.878515Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 1
2025-11-27T14:52:23.878518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.878526Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 2
2025-11-27T14:52:23.878736Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 28 partNo 2 FormedBlobsCount 0 NewHead: Offset 28 PartNo 0 PackedSize 1048790 count 1 nextOffset 29 batches 3
2025-11-27T14:52:23.878757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.878765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.878769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.878843Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 28,1 HeadOffset 28 endOffset 28 curOffset 29 d0000000000_00000000000000000028_00000_0000000001_00002? size 1048780 WTime 1764255143878
2025-11-27T14:52:23.879033Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.879060Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 28 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.882056Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 28 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.882094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.882099Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.882118Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.882129Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.882141Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 28, partNo: 0, Offset: 28 is stored on disk
2025-11-27T14:52:23.882150Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.882155Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 28, partNo: 1, Offset: 28 is stored on disk
2025-11-27T14:52:23.882160Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.882165Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 28, partNo: 2, Offset: 28 is stored on disk
2025-11-27T14:52:23.882199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.882206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.882208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.882211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.882213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.882223Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.882240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.882326Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 28
    Offset: 28
    PartNo: 0
    WriteTimestampMS: 1764255143878
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-27T14:52:23.882745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473991:3490] destroyed
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.883351Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.883756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473996:3493], now have 1 active actors on pipe
2025-11-27T14:52:23.883856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.883869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.883909Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|d31ce4cd-cb0101f8-98815e92-71502f63_29 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.883922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.883927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.883932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.883934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.883938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.883946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.883948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.883952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.883965Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.883979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.884023Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|d31ce4cd-cb0101f8-98815e92-71502f63_29"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16480
2025-11-27T14:52:23.884272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473996:3493] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|d31ce4cd-cb0101f8-98815e92-71502f63_29"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 29
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.888387Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.888893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953473999:3494], now have 1 active actors on pipe
2025-11-27T14:52:23.890052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.890074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.890203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 29 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.890283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 29 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.890300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 29 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.890305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 29 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.890322Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.890336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.890344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.890350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.890353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.890357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.890359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.890363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.890383Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 0
2025-11-27T14:52:23.890401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.890405Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 1
2025-11-27T14:52:23.890409Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.890413Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 2
2025-11-27T14:52:23.890654Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 29 partNo 2 FormedBlobsCount 0 NewHead: Offset 29 PartNo 0 PackedSize 1048790 count 1 nextOffset 30 batches 3
2025-11-27T14:52:23.890675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.890678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.890681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.890748Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 29,1 HeadOffset 29 endOffset 29 curOffset 30 d0000000000_00000000000000000029_00000_0000000001_00002? size 1048780 WTime 1764255143890
2025-11-27T14:52:23.891094Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.891115Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 29 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.893878Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 29 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.893905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.893928Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.893934Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.893936Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.893951Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 29, partNo: 0, Offset: 29 is stored on disk
2025-11-27T14:52:23.893955Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.893959Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 29, partNo: 1, Offset: 29 is stored on disk
2025-11-27T14:52:23.893963Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.893968Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 29, partNo: 2, Offset: 29 is stored on disk
2025-11-27T14:52:23.893999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.894008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.894010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.894012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.894014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.894023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.894038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.894138Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.894510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953473999:3494] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 29
    Offset: 29
    PartNo: 0
    WriteTimestampMS: 1764255143890
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.894794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.894806Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.894809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.894812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.894813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.895137Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.895576Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953474004:3497], now have 1 active actors on pipe
2025-11-27T14:52:23.895655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.895668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.895700Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f1685d46-800a1a7e-f72e7012-82f48043_30 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.895712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.895715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.895719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.895721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.895725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.895733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.895740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.895742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.895752Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.895763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.895820Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f1685d46-800a1a7e-f72e7012-82f48043_30"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:16480
2025-11-27T14:52:23.896048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953474004:3497] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f1685d46-800a1a7e-f72e7012-82f48043_30"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 30
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.902428Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.903006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953474007:3498], now have 1 active actors on pipe
2025-11-27T14:52:23.904286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.904310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.904420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 30 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.904503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 30 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.904524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 30 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.904528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 30 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.904550Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.904571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.904578Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.904585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.904588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.904592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.904594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.904600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.904623Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 0
2025-11-27T14:52:23.904641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.904649Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 1
2025-11-27T14:52:23.904653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.904656Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 2
2025-11-27T14:52:23.904884Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 30 partNo 2 FormedBlobsCount 0 NewHead: Offset 30 PartNo 0 PackedSize 1048790 count 1 nextOffset 31 batches 3
2025-11-27T14:52:23.904905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.904909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.904912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.904987Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 30,1 HeadOffset 30 endOffset 30 curOffset 31 d0000000000_00000000000000000030_00000_0000000001_00002? size 1048780 WTime 1764255143904
2025-11-27T14:52:23.905352Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.905375Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 30 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.908412Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 30 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.908441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.908466Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.908468Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.908474Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.908486Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 30, partNo: 0, Offset: 30 is stored on disk
2025-11-27T14:52:23.908491Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.908496Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 30, partNo: 1, Offset: 30 is stored on disk
2025-11-27T14:52:23.908501Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.908512Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 30, partNo: 2, Offset: 30 is stored on disk
2025-11-27T14:52:23.908551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.908559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.908561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.908563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.908567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.908576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.908594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.908707Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 30
    Offset: 30
    PartNo: 0
    WriteTimestampMS: 1764255143904
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:16480
2025-11-27T14:52:23.909054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953474007:3498] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.909674Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.910090Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953474012:3501], now have 1 active actors on pipe
2025-11-27T14:52:23.910192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.910205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.910237Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|cc790fbb-264bd77a-b95c1652-61fa14e2_31 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.910249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.910252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.910257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.910259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.910263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.910271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.910274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.910276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.910287Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.910303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.910358Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|cc790fbb-264bd77a-b95c1652-61fa14e2_31"
    Status: Active
    SeqNo: 0
  }
}
2025-11-27T14:52:23.910580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953474012:3501] destroyed
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|cc790fbb-264bd77a-b95c1652-61fa14e2_31"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 31
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:23.915682Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.916148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953474015:3502], now have 1 active actors on pipe
2025-11-27T14:52:23.917392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.917422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.917533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 31 partNo : 0 messageNo: 0 size: 511959
2025-11-27T14:52:23.917610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 31 partNo : 1 messageNo: 0 size: 511959
2025-11-27T14:52:23.917629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1976: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 31 partNo : 2 messageNo: 0 size: 24658
2025-11-27T14:52:23.917634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 31 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-27T14:52:23.917660Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.917682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.917689Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.917699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.917702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.917706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-27T14:52:23.917709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.917715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.917743Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 0
2025-11-27T14:52:23.917760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.917764Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 1
2025-11-27T14:52:23.917768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.917771Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 2
2025-11-27T14:52:23.917981Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 31 partNo 2 FormedBlobsCount 0 NewHead: Offset 31 PartNo 0 PackedSize 1048790 count 1 nextOffset 32 batches 3
2025-11-27T14:52:23.918002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-27T14:52:23.918004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-27T14:52:23.918007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.918084Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 31,1 HeadOffset 31 endOffset 31 curOffset 32 d0000000000_00000000000000000031_00000_0000000001_00002? size 1048780 WTime 1764255143917
2025-11-27T14:52:23.918239Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.918258Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 31 partNo 0 count 1 size 1048780
2025-11-27T14:52:23.921209Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 31 count 1 size 1048780 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.921237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.921261Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.921261Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.921272Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.921284Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 31, partNo: 0, Offset: 31 is stored on disk
2025-11-27T14:52:23.921289Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.921296Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 31, partNo: 1, Offset: 31 is stored on disk
2025-11-27T14:52:23.921302Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.921307Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 31, partNo: 2, Offset: 31 is stored on disk
2025-11-27T14:52:23.921340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.921349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.921351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.921354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.921356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.921367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000024_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.921370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000025_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.921373Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000026_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.921376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000027_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.921380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000028_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.921382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000029_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.921385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000030_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.921388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000031_00000_0000000001_00002? (1048780)
2025-11-27T14:52:23.921390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 8 keys were taken away. Let's read 8390240 bytes
2025-11-27T14:52:23.921408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.921422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 8 blobs
2025-11-27T14:52:23.921429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000024_00000_0000000001_00002?
2025-11-27T14:52:23.921440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000025_00000_0000000001_00002?
2025-11-27T14:52:23.921454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000026_00000_0000000001_00002?
2025-11-27T14:52:23.921468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000027_00000_0000000001_00002?
2025-11-27T14:52:23.921480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000028_00000_0000000001_00002?
2025-11-27T14:52:23.921504Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000029_00000_0000000001_00002?
2025-11-27T14:52:23.921518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000030_00000_0000000001_00002?
2025-11-27T14:52:23.921529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000031_00000_0000000001_00002?
2025-11-27T14:52:23.921543Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 8 blobs for compaction
2025-11-27T14:52:23.921506Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.921558Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 24 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.921567Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 25 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.921572Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 26 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.921578Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 27 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.921582Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 28 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.921586Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 29 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.921590Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 30 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.921598Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 31 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.921610Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 8 blobs are from cache.
2025-11-27T14:52:23.921627Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.921638Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.921642Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.921644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-27T14:52:23.921647Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.921650Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.921651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/8] d0000000000_00000000000000000024_00000_0000000001_00002?
2025-11-27T14:52:23.921653Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.921653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.921656Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.921656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000024_00000_0000000001_00002?, parameters.CurOffset=24
2025-11-27T14:52:23.921660Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 2 suffix '63'
2025-11-27T14:52:23.921824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 24.0/3
2025-11-27T14:52:23.921843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 0
2025-11-27T14:52:23.921848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 24:0 appended
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 31
    Offset: 31
    PartNo: 0
    WriteTimestampMS: 1764255143917
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
>>>>> 1
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdRead {
    ClientId: "user"
    Offset: 0
    Count: 16
    ReadTimestampMs: 0
  }
}
2025-11-27T14:52:23.922010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 24.1/3
2025-11-27T14:52:23.922024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 1
2025-11-27T14:52:23.922027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 24:1 appended
2025-11-27T14:52:23.922044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 24.2/3
2025-11-27T14:52:23.922052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 2
2025-11-27T14:52:23.922281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 24 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 1048790 count 1 nextOffset 25 batches 3
2025-11-27T14:52:23.922299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 24:2 appended
2025-11-27T14:52:23.922303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/8] d0000000000_00000000000000000025_00000_0000000001_00002?
2025-11-27T14:52:23.922305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.922308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000025_00000_0000000001_00002?, parameters.CurOffset=25
2025-11-27T14:52:23.922451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 25.0/3
2025-11-27T14:52:23.922468Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 0
2025-11-27T14:52:23.922471Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 25:0 appended
2025-11-27T14:52:23.922624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 25.1/3
2025-11-27T14:52:23.922639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 1
2025-11-27T14:52:23.922641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 25:1 appended
2025-11-27T14:52:23.922610Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.922660Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 25.2/3
2025-11-27T14:52:23.922668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 2
2025-11-27T14:52:23.922872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 25 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 2097545 count 2 nextOffset 26 batches 5
2025-11-27T14:52:23.922892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 25:2 appended
2025-11-27T14:52:23.922896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/8] d0000000000_00000000000000000026_00000_0000000001_00002?
2025-11-27T14:52:23.922898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.922901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000026_00000_0000000001_00002?, parameters.CurOffset=26
2025-11-27T14:52:23.923051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 26.0/3
2025-11-27T14:52:23.923065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 0
2025-11-27T14:52:23.923068Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 26:0 appended
2025-11-27T14:52:23.923403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 26.1/3
2025-11-27T14:52:23.923417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 1
2025-11-27T14:52:23.923420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 26:1 appended
2025-11-27T14:52:23.923452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 26.2/3
2025-11-27T14:52:23.923455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 2
2025-11-27T14:52:23.923839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 26 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 3146300 count 3 nextOffset 27 batches 7
2025-11-27T14:52:23.923858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 26:2 appended
2025-11-27T14:52:23.923863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/8] d0000000000_00000000000000000027_00000_0000000001_00002?
2025-11-27T14:52:23.923864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.923867Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000027_00000_0000000001_00002?, parameters.CurOffset=27
2025-11-27T14:52:23.923995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 27.0/3
2025-11-27T14:52:23.924008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 0
2025-11-27T14:52:23.924011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 27:0 appended
2025-11-27T14:52:23.924160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 27.1/3
2025-11-27T14:52:23.924173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 1
2025-11-27T14:52:23.924175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 27:1 appended
2025-11-27T14:52:23.924189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 27.2/3
2025-11-27T14:52:23.924191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 2
2025-11-27T14:52:23.924580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 27 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 4195055 count 4 nextOffset 28 batches 9
2025-11-27T14:52:23.924600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 27:2 appended
2025-11-27T14:52:23.924605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/8] d0000000000_00000000000000000028_00000_0000000001_00002?
2025-11-27T14:52:23.924607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.924611Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000028_00000_0000000001_00002?, parameters.CurOffset=28
2025-11-27T14:52:23.924751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 28.0/3
2025-11-27T14:52:23.924765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 0
2025-11-27T14:52:23.924769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 28:0 appended
2025-11-27T14:52:23.924914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 28.1/3
2025-11-27T14:52:23.924928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 1
2025-11-27T14:52:23.924930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 28:1 appended
2025-11-27T14:52:23.924943Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 28.2/3
2025-11-27T14:52:23.924950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 2
2025-11-27T14:52:23.925138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 28 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 5243810 count 5 nextOffset 29 batches 11
2025-11-27T14:52:23.925153Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 28:2 appended
2025-11-27T14:52:23.925158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/8] d0000000000_00000000000000000029_00000_0000000001_00002?
2025-11-27T14:52:23.925160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.925162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000029_00000_0000000001_00002?, parameters.CurOffset=29
2025-11-27T14:52:23.925298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 29.0/3
2025-11-27T14:52:23.925310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 0
2025-11-27T14:52:23.925313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 29:0 appended
2025-11-27T14:52:23.925462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 29.1/3
2025-11-27T14:52:23.925473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 1
2025-11-27T14:52:23.925476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 29:1 appended
2025-11-27T14:52:23.925488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 29.2/3
2025-11-27T14:52:23.925493Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 2
2025-11-27T14:52:23.925687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 29 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 6292565 count 6 nextOffset 30 batches 13
2025-11-27T14:52:23.925700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 29:2 appended
2025-11-27T14:52:23.925703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/8] d0000000000_00000000000000000030_00000_0000000001_00002?
2025-11-27T14:52:23.925707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.925709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000030_00000_0000000001_00002?, parameters.CurOffset=30
2025-11-27T14:52:23.925869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 30.0/3
2025-11-27T14:52:23.925882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 0
2025-11-27T14:52:23.925884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 30:0 appended
2025-11-27T14:52:23.926039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 30.1/3
2025-11-27T14:52:23.926051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 1
2025-11-27T14:52:23.926053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 30:1 appended
2025-11-27T14:52:23.926065Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 30.2/3
2025-11-27T14:52:23.926073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 2
2025-11-27T14:52:23.927518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 30 partNo 2 result is x0000000000_00000000000000000023_00000_0000000007_00016 size 8365364
2025-11-27T14:52:23.927540Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1104: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000023_00000_0000000007_00016 new key d0000000000_00000000000000000023_00000_0000000007_00016 size 8365364 WTime 1764255143927
2025-11-27T14:52:23.927567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 30 partNo 2 FormedBlobsCount 1 NewHead: Offset 30 PartNo 2 PackedSize 24736 count 1 nextOffset 31 batches 1
2025-11-27T14:52:23.927573Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 30:2 appended
2025-11-27T14:52:23.927577Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/8] d0000000000_00000000000000000031_00000_0000000001_00002?
2025-11-27T14:52:23.927579Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.927582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000031_00000_0000000001_00002?, parameters.CurOffset=31
2025-11-27T14:52:23.927737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 31.0/3
2025-11-27T14:52:23.927752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 0
2025-11-27T14:52:23.927755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 31:0 appended
2025-11-27T14:52:23.927912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 31.1/3
2025-11-27T14:52:23.927925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 1
2025-11-27T14:52:23.927927Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 31:1 appended
2025-11-27T14:52:23.927938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 31.2/3
2025-11-27T14:52:23.927946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 2
2025-11-27T14:52:23.928155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 31 partNo 2 FormedBlobsCount 0 NewHead: Offset 30 PartNo 2 PackedSize 1073491 count 2 nextOffset 32 batches 3
2025-11-27T14:52:23.928168Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 31:2 appended
2025-11-27T14:52:23.928222Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 30,2 HeadOffset 23 endOffset 24 curOffset 32 d0000000000_00000000000000000030_00002_0000000002_00002| size 1073481 WTime 1764255143928
2025-11-27T14:52:23.928380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953474015:3502] destroyed
2025-11-27T14:52:23.928396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418142953474020:3505], now have 1 active actors on pipe
2025-11-27T14:52:23.928423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.928434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.928450Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.928465Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 23 partNo 0 count 7 size 8365364
2025-11-27T14:52:23.928477Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 30 partNo 2 count 2 size 1073481
2025-11-27T14:52:23.928566Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 4 Topic 'rt3.dc1--topic1' partition 0 user user offset 0 partno 0 count 16 size 26214400 endOffset 32 max time lag 0ms effective offset 0
2025-11-27T14:52:23.928774Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 4 added 3 blobs, size 25145475 count 24 last offset 15, current partition end offset: 32
2025-11-27T14:52:23.928794Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 4. Send blob request.
2025-11-27T14:52:23.928808Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 7 parts_count 16 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.928814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 7 partno 2 count 8 parts_count 15 source 1 size 7878063 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.928819Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 15 partno 1 count 8 parts_count 15 source 1 size 7878063 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.928836Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 4. Have to read 1 of 3 from KV
2025-11-27T14:52:23.928863Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 7 partno 2 count 8 parts 15 suffix '0'
2025-11-27T14:52:23.928876Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 15 partno 1 count 8 parts 15 suffix '0'
2025-11-27T14:52:23.928881Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 7 parts_count 16
2025-11-27T14:52:23.945100Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 3 from KV. Status 1
2025-11-27T14:52:23.945122Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-11-27T14:52:23.945138Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 7 size 8365339 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.945352Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 3 blobs
2025-11-27T14:52:23.945365Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 2 suffix '124' size 1048780
2025-11-27T14:52:23.945371Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 7 parts 16 suffix '0' size 8365339
2025-11-27T14:52:23.945561Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:23.945721Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:23.945956Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-27T14:52:23.946107Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 1 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:23.946271Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 1 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-27T14:52:23.946603Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 2 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:23.946772Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 2 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-27T14:52:23.947292Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 3 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:23.947562Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 3 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-27T14:52:23.947739Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 4 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:23.947931Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 4 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-27T14:52:23.948099Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 5 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:23.948478Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 5 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-27T14:52:23.948636Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 6 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:23.948670Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 6 totakecount 7 count 1 size 24704 from pos 0 cbcount 1
2025-11-27T14:52:23.948947Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 7 totakecount 7 count 0 size 1024007 from pos 0 cbcount 2
2025-11-27T14:52:23.948991Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 7 totakecount 8 count 1 size 24704 from pos 0 cbcount 1
2025-11-27T14:52:23.949143Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 8 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:23.949350Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 8 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:23.949548Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 8 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-27T14:52:23.949890Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 9 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:23.950071Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 9 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-27T14:52:23.950252Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 10 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:23.950438Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 10 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-27T14:52:23.950608Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 11 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:23.950798Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 11 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-27T14:52:23.950965Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 12 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:23.951157Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 12 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-27T14:52:23.951314Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 13 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:23.951508Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 13 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-27T14:52:23.951667Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 14 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:23.951692Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 14 totakecount 8 count 1 size 24704 from pos 0 cbcount 1
2025-11-27T14:52:23.951844Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 15 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:23.952033Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 15 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:23.952049Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 15 totakecount 8 count 1 size 24704 from pos 0 cbcount 1
2025-11-27T14:52:23.952156Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 23 size 1048780 cause it's been evicted from L2. Actual L1 size: 11
2025-11-27T14:52:23.952163Z node 2 :PERSQUEUE NOTICE: read.h:372: [72075186224037892][PQCacheProxy]Have to remove new data from cache. Topic rt3.dc1--topic1, tablet id72075186224037892, cookie 0
2025-11-27T14:52:23.952259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.952293Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 23 count 7 size 8365364 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.952299Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 30 count 2 size 1073481 actorID [2:7577418141312432496:2363]
2025-11-27T14:52:23.952370Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.952376Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.952381Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.952386Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.952391Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.952398Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.952402Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.952407Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 7 parts 16 suffix '0' size 8365364
2025-11-27T14:52:23.952414Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-27T14:52:23.952421Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 7 partno 2 count 8 parts 15 suffix '0' size 7878063
2025-11-27T14:52:23.952425Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 30 partno 2 count 2 parts 2 suffix '124' size 1073481
2025-11-27T14:52:23.967693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.967917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.967933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.967936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.967938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.967940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.967970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.967988Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 29 size 1048780 cause it's been evicted from L2. Actual L1 size: 12
2025-11-27T14:52:23.967992Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 26 size 1048780 cause it's been evicted from L2. Actual L1 size: 11
2025-11-27T14:52:23.967999Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 30 size 1048780 cause it's been evicted from L2. Actual L1 size: 10
2025-11-27T14:52:23.968003Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 25 size 1048780 cause it's been evicted from L2. Actual L1 size: 9
2025-11-27T14:52:23.968007Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 27 size 1048780 cause it's been evicted from L2. Actual L1 size: 8
2025-11-27T14:52:23.968011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 31 size 1048780 cause it's been evicted from L2. Actual L1 size: 7
2025-11-27T14:52:23.968014Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 7 size 7878063 cause it's been evicted from L2. Actual L1 size: 6
2025-11-27T14:52:23.968018Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 24 size 1048780 cause it's been evicted from L2. Actual L1 size: 5
2025-11-27T14:52:23.968021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 28 size 1048780 cause it's been evicted from L2. Actual L1 size: 4
2025-11-27T14:52:23.968026Z node 2 :PERSQUEUE NOTICE: read.h:372: [72075186224037892][PQCacheProxy]Have to remove new data from cache. Topic rt3.dc1--topic1, tablet id72075186224037892, cookie 0
2025-11-27T14:52:23.968049Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.968055Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000024_00000_0000000001_00002?(+) to d0000000000_00000000000000000024_00000_0000000001_00002?(+)
2025-11-27T14:52:23.968072Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000025_00000_0000000001_00002?(+) to d0000000000_00000000000000000025_00000_0000000001_00002?(+)
2025-11-27T14:52:23.968074Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000026_00000_0000000001_00002?(+) to d0000000000_00000000000000000026_00000_0000000001_00002?(+)
2025-11-27T14:52:23.968079Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000027_00000_0000000001_00002?(+) to d0000000000_00000000000000000027_00000_0000000001_00002?(+)
2025-11-27T14:52:23.968085Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000028_00000_0000000001_00002?(+) to d0000000000_00000000000000000028_00000_0000000001_00002?(+)
2025-11-27T14:52:23.968087Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000029_00000_0000000001_00002?(+) to d0000000000_00000000000000000029_00000_0000000001_00002?(+)
2025-11-27T14:52:23.968090Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000030_00000_0000000001_00002?(+) to d0000000000_00000000000000000030_00000_0000000001_00002?(+)
2025-11-27T14:52:23.968092Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000031_00000_0000000001_00002?(+) to d0000000000_00000000000000000031_00000_0000000001_00002?(+)
2025-11-27T14:52:23.968094Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000023_00000_0000000001_00002|(+) to d0000000000_00000000000000000023_00000_0000000001_00002|(+)
2025-11-27T14:52:23.977009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.977045Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.977058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.977061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.977064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.977067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.977069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.977082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.982131Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.986363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418142953474020:3505] destroyed
2025-11-27T14:52:23.995155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.995172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.995175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.995179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.995183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdReadResult {
    MaxOffset: 32
    Result {
      Offset: 0
      Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
>>>>> 2
CallPersQueueGRPC request to localhost:16480
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdRead {
    ClientId: "user"
    Offset: 16
    Count: 16
    ReadTimestampMs: 0
  }
}
2025-11-27T14:52:24.045013Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.045528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147248441331:3517], now have 1 active actors on pipe
2025-11-27T14:52:24.045634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.045648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.045694Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 5 Topic 'rt3.dc1--topic1' partition 0 user user offset 16 partno 0 count 16 size 26214400 endOffset 32 max time lag 0ms effective offset 16
2025-11-27T14:52:24.045896Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 5 added 2 blobs, size 9438782 count 16 last offset 23, current partition end offset: 32
2025-11-27T14:52:24.045905Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 5. Send blob request.
2025-11-27T14:52:24.045917Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 15 partno 1 count 8 parts_count 15 source 1 size 7878063 accessed 1 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:24.045921Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 23 partno 0 count 7 parts_count 16 source 1 size 8365364 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:24.045932Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 5. All 2 blobs are from cache.
2025-11-27T14:52:24.045956Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 15 partno 1 count 8 parts 15 suffix '0'
2025-11-27T14:52:24.045967Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 7 parts 16 suffix '0'
2025-11-27T14:52:24.045970Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 2 blobs
2025-11-27T14:52:24.046404Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 16 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:24.046645Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 16 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:24.046786Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 16 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-27T14:52:24.046969Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 17 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:24.047121Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 17 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-27T14:52:24.047281Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 18 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:24.047428Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 18 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-27T14:52:24.047588Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 19 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:24.047739Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 19 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-27T14:52:24.047863Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 20 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:24.048006Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 20 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-27T14:52:24.048211Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 21 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:24.048358Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 21 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-27T14:52:24.048579Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 22 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:24.048598Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 22 totakecount 8 count 1 size 24704 from pos 0 cbcount 1
2025-11-27T14:52:24.048836Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 23 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:24.049022Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 23 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:24.049040Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 23 totakecount 7 count 1 size 24704 from pos 0 cbcount 1
2025-11-27T14:52:24.049215Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 24 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:24.049350Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 24 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:24.049503Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 24 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-27T14:52:24.049644Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 25 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:24.049791Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 25 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-27T14:52:24.049925Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 26 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:24.050076Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 26 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-27T14:52:24.050214Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 27 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:24.050369Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 27 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-27T14:52:24.050777Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 28 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:24.050955Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 28 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-27T14:52:24.051454Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 29 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-27T14:52:24.051485Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 29 totakecount 7 count 1 size 24704 from pos 0 cbcount 1
2025-11-27T14:52:24.051789Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 30 totakecount 7 count 0 size 1024007 from pos 0 cbcount 2
2025-11-27T14:52:24.051891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.068531Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.077300Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147248441331:3517] destroyed
CallPersQueueGRPC response:
2025-11-27T14:52:24.095503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.095520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.095523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.095526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.095528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdReadResult {
    MaxOffset: 32
    Result {
      Offset: 16
      Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-27T14:52:24.195882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.195904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.195907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.195911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.195914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.296224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.296245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.296248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.296252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.296256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.396585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.396606Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.396610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.396614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.396617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
[[bad]]assertion failed at ydb/services/persqueue_v1/persqueue_ut.cpp:3269, virtual void NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TTestCaseCache::Execute_(NUnitTest::TTestContext &): (info0.BlobsFromCache == 1) failed: (2 != 1) [[rst]]
[[alt1]]TBackTrace::Capture()+28 (0x150BCE9C)
NUnitTest::NPrivate::RaiseError(char const*, TBasicString<char, std::__y1::char_traits<char>> const&, bool)+128 (0x15278EA0)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TTestCaseCache::Execute_(NUnitTest::TTestContext&)+4375 (0x14DE71B7)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TCurrentTest::Execute()::'lambda'()::operator()() const+71 (0x14ED3D07)
NUnitTest::TTestBase::Run(std::__y1::function<void ()>, TBasicString<char, std::__y1::char_traits<char>> const&, char const*, bool)+120 (0x1527B078)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TCurrentTest::Execute()+436 (0x14ED3604)
NUnitTest::TTestFactory::Execute()+817 (0x1527B7C1)
NUnitTest::RunMain(int, char**)+3229 (0x1528DC2D)
??+0 (0x7FB9525FAD90)
__libc_start_main+128 (0x7FB9525FAE40)
_start+41 (0x13EDB029)
[[rst]]
2025-11-27T14:52:17.331508Z node 1 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[1:7577418117231651519:2235];send_to=[0:7307199536658146131:7762515];
2025-11-27T14:52:17.331680Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
2025-11-27T14:52:17.343391Z node 2 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[2:7577418119417944838:2160];send_to=[0:7307199536658146131:7762515];
test_client.cpp: SetPath # /home/runner/.ya/build/build_root/9gmc/000348/r3tmp/tmp2ZIGPI/pdisk_1.dat
2025-11-27T14:52:17.344506Z node 1 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-11-27T14:52:17.346819Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
2025-11-27T14:52:17.346865Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-11-27T14:52:17.392473Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-11-27T14:52:17.393072Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-11-27T14:52:17.412815Z node 1 :IMPORT WARN: schemeshard_import.cpp:307: Table profiles were not loaded
2025-11-27T14:52:17.430292Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-11-27T14:52:17.430343Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Disconnected -> Connecting
TServer::EnableGrpc on GrpcPort 6400, node 1
2025-11-27T14:52:17.432583Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-11-27T14:52:17.445060Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-11-27T14:52:17.445095Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-11-27T14:52:17.446631Z node 1 :HIVE WARN: hive_impl.cpp:811: HIVE#72057594037968897 Handle TEvInterconnect::TEvNodeConnected, NodeId 2 Cookie 2
2025-11-27T14:52:17.446952Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-11-27T14:52:17.499177Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:190: distributable config is empty, broken or outdated, will use file: /home/runner/.ya/build/build_root/9gmc/000348/r3tmp/yandexmHhS5p.tmp
2025-11-27T14:52:17.499194Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:196: will try to initialize from file: /home/runner/.ya/build/build_root/9gmc/000348/r3tmp/yandexmHhS5p.tmp
2025-11-27T14:52:17.499278Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:202: successfully initialized from file: /home/runner/.ya/build/build_root/9gmc/000348/r3tmp/yandexmHhS5p.tmp
2025-11-27T14:52:17.499313Z node 1 :NET_CLASSIFIER ERROR: net_classifier.cpp:228: got bad distributable configuration
2025-11-27T14:52:17.555038Z INFO: TTestServer started on Port 25348 GrpcPort 6400
2025-11-27T14:52:17.564940Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
TClient is connected to server localhost:25348
PQClient connected to localhost:6400
WaitRootIsUp 'Root'...
TClient::Ls request: Root
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "Root"
    PathId: 1
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 1
    CreateStep: 0
    ParentPathId: 1
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 1
      SubDomainVersion: 0
      SecurityStateVersion: 0
    }
    ChildrenExist: false
  }
  Children {
    Name: ".sys"
    PathId: 18446744073709551615
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 0
    CreateStep: 0
    ParentPathId: 18446744073709551615
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 0
      Pl...
(TRUNCATED)
WaitRootIsUp 'Root' success.
2025-11-27T14:52:17.589081Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpAlterSubDomain, opId: 281474976720657:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_alter_subdomain.cpp:311)
waiting...
waiting...
2025-11-27T14:52:17.608311Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720659:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
waiting...
2025-11-27T14:52:17.620993Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
waiting...
waiting...
2025-11-27T14:52:17.843616Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7577418119417945113:2300], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-27T14:52:17.843617Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7577418119417945121:2303], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-27T14:52:17.843643Z node 2 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-27T14:52:17.843781Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7577418119417945128:2305], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-27T14:52:17.843812Z node 2 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-27T14:52:17.845432Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateResourcePool, opId: 281474976715657:3, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:179)
2025-11-27T14:52:17.851520Z node 2 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [2:7577418119417945127:2304], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976715657 completed, doublechecking }
2025-11-27T14:52:17.966218Z node 2 :TX_PROXY ERROR: schemereq.cpp:593: Actor# [2:7577418119417945157:2174] txid# 281474976715658, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 9], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-11-27T14:52:18.005454Z node 1 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:625: Compilation failed, self: [1:7577418117231652390:2331], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-11-27T14:52:18.005525Z node 2 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:625: Compilation failed, self: [2:7577418119417945171:2309], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-11-27T14:52:18.005691Z node 2 :KQP_SESSION WARN: kqp_session_actor.cpp:2542: SessionId: ydb://session/3?node_id=2&id=YTlmYjc0OTEtMmEwZjNkY2ItOTZkZmUyOGEtN2IxOTc0MWI=, ActorId: [2:7577418119417945111:2299], ActorState: ExecuteState, TraceId: 01kb2wt01j449xkcxcwccxc1n9, ReplyQueryCompileError, status: SCHEME_ERROR, issues: { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } }, remove tx with tx_id: 
2025-11-27T14:52:18.006155Z node 2 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-11-27T14:52:18.008963Z node 1 :KQP_SESSION WARN: kqp_session_actor.cpp:2542: SessionId: ydb://session/3?node_id=1&id=NjI0MzQwZDEtODc2ZDI1NjEtMzdjMGExZTItZWMyYzZhNA==, ActorId: [1:7577418117231652331:2322], ActorState: ExecuteState, TraceId: 01kb2wt01q35zkccpmctgabpe0, ReplyQueryCompileError, status: SCHEME_ERROR, issues: { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } }, remove tx with tx_id: 
2025-11-27T14:52:18.009075Z node 1 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-11-27T14:52:18.027511Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720662:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
2025-11-27T14:52:18.042681Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720663:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
2025-11-27T14:52:18.070208Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720664:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
=== Init DC: UPSERT INTO `/Root/PQ/Config/V2/Cluster` (name, balancer, local, enabled, weight) VALUES ("dc1", "localhost", true, true, 1000), ("dc2", "dc2.logbroker.yandex.net", false, true, 1000);
=== CheckClustersList. Subcribe to ClusterTracker from [1:7577418121526620145:3033] 
2025-11-27T14:52:18.331804Z node 1 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-11-27T14:52:18.346190Z node 2 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-11-27T14:52:22.331506Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[1:7577418117231651519:2235];send_to=[0:7307199536658146131:7762515];
2025-11-27T14:52:22.331538Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
2025-11-27T14:52:22.343824Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[2:7577418119417944838:2160];send_to=[0:7307199536658146131:7762515];
2025-11-27T14:52:22.343864Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
=== CheckClustersList. Ok
PQ Client: create topic: rt3.dc1--topic1 with 1 partitions
CallPersQueueGRPC request to localhost:6400
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--topic1"
  }
}
2025-11-27T14:52:23.241943Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--topic1, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:6400
MetaRequest {
  CmdCreateTopic {
    Topic: "rt3.dc1--topic1"
    NumPartitions: 1
    Config {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 6291456
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
  }
}
CallPersQueueGRPC response:
Status: 129
ProxyErrorCode: 53
SchemeStatus: 1
FlatTxId {
  TxId: 281474976720671
  SchemeShardTabletId: 72057594046644480
  PathId: 13
}
ErrorCode: OK
AddTopic: rt3.dc1--topic1
===Run query:``DECLARE $version as Int64; DECLARE $path AS Utf8; DECLARE $cluster as Utf8; UPSERT INTO `/Root/PQ/Config/V2/Topics` (path, dc) VALUES ($path, $cluster); UPSERT INTO `/Root/PQ/Config/V2/Versions` (name, version) VALUES ("Topics", $version);`` with topic = topic1, dc = dc1
2025-11-27T14:52:23.251369Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1656: [72075186224037893][] pipe [1:7577418143001457055:3269] connected; active server actors: 1
2025-11-27T14:52:23.251571Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1520: [72075186224037893][rt3.dc1--topic1] updating configuration. Deleted partitions []. Added partitions [0]
2025-11-27T14:52:23.251703Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:971: [72075186224037893][rt3.dc1--topic1] Discovered subdomain [OwnerId: 72057594046644480, LocalPathId: 1] state, outOfSpace = 0 at RB 72075186224037893
2025-11-27T14:52:23.252712Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:140: [72075186224037893][rt3.dc1--topic1] BALANCER INIT DONE for rt3.dc1--topic1: (0, 72075186224037892) 
2025-11-27T14:52:23.257311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2959: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-11-27T14:52:23.259816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2991: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-11-27T14:52:23.259967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:787: [PQ: 72075186224037892] no config, start with empty partitions and default config
2025-11-27T14:52:23.259999Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:913: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-11-27T14:52:23.260015Z node 2 :PERSQUEUE INFO: pq_impl.cpp:612: [PQ: 72075186224037892] doesn't have tx writes info
2025-11-27T14:52:23.260150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457076:2454], now have 1 active actors on pipe
2025-11-27T14:52:23.260160Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457054:3268], now have 1 active actors on pipe
2025-11-27T14:52:23.260296Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:404: [72075186224037893][rt3.dc1--topic1] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1
2025-11-27T14:52:23.261520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3563: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-27T14:52:23.262383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-27T14:52:23.264567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457112:3315], now have 1 active actors on pipe
2025-11-27T14:52:23.269059Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitConfigStep
2025-11-27T14:52:23.270115Z node 2 :PERSQUEUE DEBUG: partition.cpp:1315: [72075186224037892][Partition][0][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-11-27T14:52:23.270266Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-11-27T14:52:23.270373Z node 2 :PERSQUEUE INFO: partition_init.cpp:1107: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7577418145187749211:2363]
2025-11-27T14:52:23.272016Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitFieldsStep
2025-11-27T14:52:23.272035Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--topic1:0:Initializer] Initializing completed.
2025-11-27T14:52:23.272042Z node 2 :PERSQUEUE INFO: partition.cpp:707: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--topic1' partition 0 generation 1 [2:7577418145187749211:2363]
2025-11-27T14:52:23.272075Z node 2 :PERSQUEUE DEBUG: partition.cpp:727: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--topic1 partitition 0 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-11-27T14:52:23.272095Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:107: [72075186224037892][Partition][0][StateIdle] Initializing MLP Consumers: 0
2025-11-27T14:52:23.272447Z node 2 :PERSQUEUE DEBUG: partition.cpp:4404: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 1
2025-11-27T14:52:23.272476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.272486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2392: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TTransaction[ProposeConfig])
2025-11-27T14:52:23.272561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.272569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.272572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.272574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.272593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.272599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.272601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.272603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.272605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.272615Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:930: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-11-27T14:52:23.272626Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:70: [72075186224037892][Partition][0][StateIdle] Process MLP pending events. Count 0
2025-11-27T14:52:23.272734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3563: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-27T14:52:23.272765Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.275972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-27T14:52:23.276074Z node 2 :PERSQUEUE DEBUG: partition.cpp:1420: [72075186224037892][Partition][0][StateIdle] Handle TEvPQ::TEvTxCommit Step 1764255143308, TxId 281474976720671
2025-11-27T14:52:23.276081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.276086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.276088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.276098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2456: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TTransaction[ProposeConfig])
2025-11-27T14:52:23.276125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.276127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.276130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.276186Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.277361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.277425Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:107: [72075186224037892][Partition][0][StateIdle] Initializing MLP Consumers: 0
2025-11-27T14:52:23.277468Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.277476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.277480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.277483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.277486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.277489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.277500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.277631Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:398: [PQ: 72075186224037892] Apply new config PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 6291456 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--topic1" Version: 0 LocalDC: true Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } TopicPath: "/Root/PQ/rt3.dc1--topic1" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Version: 0 Generation: 0 } MonitoringProjectId: ""
2025-11-27T14:52:23.277658Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:913: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-11-27T14:52:23.277718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3563: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-27T14:52:23.278799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-27T14:52:23.278841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3563: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-27T14:52:23.279756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
===Query complete
TClient::Ls request: /Root/PQ/rt3.dc1--topic1
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--topic1"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720671
    CreateStep: 1764255143308
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--topic1"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 6291456
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 600000...
(TRUNCATED)
GetTopicVersionFromPath:  record Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--topic1"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720671
    CreateStep: 1764255143308
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--topic1"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 6291456
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      YdbDatabasePath: "/Root"
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
    Partitions {
      PartitionId: 0
      TabletId: 72075186224037892
      Status: Active
    }
    AlterVersion: 1
    BalancerTabletID: 72075186224037893
    NextPartitionId: 1
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 1
      PlanResolution: 7
      Coordinators: 72057594046316545
      TimeCastBucketsPerMediator: 2
      Mediators: 72057594046382081
    }
    DomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    PathsInside: 12
    PathsLimit: 10000
    ShardsInside: 6
    ShardsLimit: 200000
    ResourcesDomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    DiskSpaceUsage {
      Tables {
        TotalSize: 0
        DataSize: 0
        IndexSize: 0
      }
      Topics {
        ReserveSize: 0
        AccountSize: 0
        DataSize: 0
        UsedReserveSize: 0
      }
    }
    PQPartitionsInside: 1
    PQPartitionsLimit: 1000000
    SchemeLimits {
      MaxDepth: 32
      MaxPaths: 10000
      MaxChildrenInDir: 100000
      MaxAclBytesSize: 10240
      MaxTableColumns: 200
      MaxTableColumnNameLength: 255
      MaxTableKeyColumns: 30
      MaxTableIndices: 20
      MaxShards: 200000
      MaxShardsInPath: 35000
      MaxConsistentCopyTargets: 10000
      MaxPathElementLength: 255
      ExtraPathSymbolsAllowed: "!\"#$%&\'()*+,-.:;<=>?@[\\]^_`{|}~"
      MaxPQPartitions: 1000000
      MaxTableCdcStreams: 5
      MaxExports: 10
      MaxImports: 10
      MaxColumnTableColumns: 10000
    }
  }
}
Path: "/Root/PQ/rt3.dc1--topic1"
 name rt3.dc1--topic1 version1
CallPersQueueGRPC request to localhost:6400
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--topic1"
  }
}
2025-11-27T14:52:23.286666Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
MetaResponse {
  CmdGetTopicMetadataResult {
    TopicInfo {
      Topic: "rt3.dc1--topic1"
      NumPartitions: 1
      Config {
        PartitionConfig {
          LifetimeSeconds: 86400
          LowWatermark: 6291456
          SourceIdLifetimeSeconds: 86400
          WriteSpeedInBytesPerSecond: 20000000
          BurstSize: 20000000
          SourceIdMaxCounts: 6000000
        }
        Version: 1
        LocalDC: true
        Codecs {
          Ids: 0
          Ids: 1
          Ids: 2
          Codecs: "raw"
          Codecs: "gzip"
          Codecs: "lzop"
        }
        TopicPath: "/Root/PQ/rt3.dc1--topic1"
        YdbDatabasePath: "/Root"
        Consumers {
          Name: "user"
          ReadFromTimestampsMs: 0
          FormatVersion: 0
          Version: 0
        }
      }
      ErrorCode: OK
    }
  }
}
=== Topic created, have version: 1
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.287737Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.288231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457167:3348], now have 1 active actors on pipe
2025-11-27T14:52:23.288297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.288326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.288363Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|52b21d6a-7b854588-b483adca-8fc8b48b_0 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.288381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.288390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.288397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.288401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.288406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.288422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.288428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.288432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.288449Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.288464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.288543Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.288760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457167:3348] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|52b21d6a-7b854588-b483adca-8fc8b48b_0"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|52b21d6a-7b854588-b483adca-8fc8b48b_0"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 0
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.292400Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.292846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457170:3349], now have 1 active actors on pipe
2025-11-27T14:52:23.293269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.293283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.293381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 0 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.293411Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.293432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.293440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.293453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.293459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.293462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.293491Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 0
2025-11-27T14:52:23.293554Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 0 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 262214 count 1 nextOffset 1 batches 1
2025-11-27T14:52:23.293569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.293572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.293576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.293848Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 0,1 HeadOffset 0 endOffset 0 curOffset 1 d0000000000_00000000000000000000_00000_0000000001_00000? size 262204 WTime 1764255143293
2025-11-27T14:52:23.293896Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.293912Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 1 size 262204
2025-11-27T14:52:23.296085Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 0 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.296111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.296127Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.296134Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.296142Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.296151Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 0, partNo: 0, Offset: 0 is stored on disk
2025-11-27T14:52:23.296188Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:930: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-11-27T14:52:23.296199Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:972: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-11-27T14:52:23.296202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.296205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.296207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.296209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.296211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.296218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.296230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.296267Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--topic1' partition 0 user user offset 0 partno 0 count 1 size 1024000 endOffset 1 max time lag 0ms effective offset 0
2025-11-27T14:52:23.296285Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 262204 count 1 last offset 0, current partition end offset: 1
2025-11-27T14:52:23.296292Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-11-27T14:52:23.296315Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.296326Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 3. All 1 blobs are from cache.
2025-11-27T14:52:23.296332Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.296339Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 1 blobs
2025-11-27T14:52:23.296415Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 1 count 1 size 262182 from pos 0 cbcount 1
2025-11-27T14:52:23.296438Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1022: Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp done, result 1764255143293 queuesize 0 startOffset 0
2025-11-27T14:52:23.296307Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.296606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457170:3349] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 0
    Offset: 0
    WriteTimestampMS: 1764255143293
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.297287Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.297592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457174:3351], now have 1 active actors on pipe
2025-11-27T14:52:23.297679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.297690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.297721Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b4ec1f8c-6b7fc3e2-e983972e-a18570d7_1 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.297733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.297736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.297740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.297747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.297750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.297758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.297766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.297769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.297782Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.297797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.297842Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b4ec1f8c-6b7fc3e2-e983972e-a18570d7_1"
    Status: Active
    SeqNo: 0
  }
}
2025-11-27T14:52:23.298046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457174:3351] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b4ec1f8c-6b7fc3e2-e983972e-a18570d7_1"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 1
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.300791Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.301106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457180:3352], now have 1 active actors on pipe
2025-11-27T14:52:23.301479Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.301494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.301516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 1 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.301531Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.301548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.301555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.301565Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.301571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.301575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.301603Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 0
2025-11-27T14:52:23.301631Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 1 partNo 0 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 262214 count 1 nextOffset 2 batches 1
2025-11-27T14:52:23.301642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.301645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.301648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.301861Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 1,1 HeadOffset 1 endOffset 1 curOffset 2 d0000000000_00000000000000000001_00000_0000000001_00000? size 262204 WTime 1764255143301
2025-11-27T14:52:23.301902Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.301919Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 1 partNo 0 count 1 size 262204
2025-11-27T14:52:23.304029Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 1 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.304051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.304081Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.304087Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.304093Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.304103Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 1, partNo: 0, Offset: 1 is stored on disk
2025-11-27T14:52:23.304140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.304147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.304149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.304151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.304153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.304160Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.304184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.304247Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 1
    Offset: 1
    WriteTimestampMS: 1764255143301
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-27T14:52:23.304469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457180:3352] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.304928Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.305291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457184:3354], now have 1 active actors on pipe
2025-11-27T14:52:23.305396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.305406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.305434Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9009c22-a804c8f5-b481a287-4d7d25ae_2 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.305442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.305444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.305448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.305450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.305452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.305460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.305462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.305465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.305474Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.305483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.305545Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9009c22-a804c8f5-b481a287-4d7d25ae_2"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9009c22-a804c8f5-b481a287-4d7d25ae_2"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 2
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.306405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457184:3354] destroyed
2025-11-27T14:52:23.307316Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.307628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457187:3355], now have 1 active actors on pipe
2025-11-27T14:52:23.307985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.307996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.308019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 2 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.308032Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.308047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.308055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.308073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.308079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.308082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.308106Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 0
2025-11-27T14:52:23.308148Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 2 partNo 0 FormedBlobsCount 0 NewHead: Offset 2 PartNo 0 PackedSize 262214 count 1 nextOffset 3 batches 1
2025-11-27T14:52:23.308158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.308161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.308163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.308393Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 2,1 HeadOffset 2 endOffset 2 curOffset 3 d0000000000_00000000000000000002_00000_0000000001_00000? size 262204 WTime 1764255143308
2025-11-27T14:52:23.308433Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.308448Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 2 partNo 0 count 1 size 262204
2025-11-27T14:52:23.310413Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 2 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.310432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.310452Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.310463Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.310467Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.310476Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 2, partNo: 0, Offset: 2 is stored on disk
2025-11-27T14:52:23.310502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.310508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.310510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.310512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.310514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.310521Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.310530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.310641Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 2
    Offset: 2
    WriteTimestampMS: 1764255143307
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-27T14:52:23.310907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457187:3355] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.311353Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.311664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457191:3357], now have 1 active actors on pipe
2025-11-27T14:52:23.311733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.311744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.311774Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a0c27e6c-de84a0d3-ef34866f-dec3e639_3 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.311786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.311789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.311793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.311795Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.311798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.311807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.311814Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.311817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.311828Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.311842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.311909Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a0c27e6c-de84a0d3-ef34866f-dec3e639_3"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a0c27e6c-de84a0d3-ef34866f-dec3e639_3"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 3
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.313008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457191:3357] destroyed
2025-11-27T14:52:23.313831Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.314327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457194:3358], now have 1 active actors on pipe
2025-11-27T14:52:23.314719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.314734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.314755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 3 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.314767Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.314783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.314791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.314801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.314803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.314806Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.314831Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 0
2025-11-27T14:52:23.314861Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 3 partNo 0 FormedBlobsCount 0 NewHead: Offset 3 PartNo 0 PackedSize 262214 count 1 nextOffset 4 batches 1
2025-11-27T14:52:23.314873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.314876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.314879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.314992Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 3,1 HeadOffset 3 endOffset 3 curOffset 4 d0000000000_00000000000000000003_00000_0000000001_00000? size 262204 WTime 1764255143314
2025-11-27T14:52:23.315034Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.315050Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 3 partNo 0 count 1 size 262204
2025-11-27T14:52:23.317193Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 3 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.317224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.317239Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.317247Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.317262Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.317276Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 3, partNo: 0, Offset: 3 is stored on disk
2025-11-27T14:52:23.317311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.317319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.317321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.317323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.317325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.317333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.317344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.317413Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.317708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457194:3358] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 3
    Offset: 3
    WriteTimestampMS: 1764255143314
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.318324Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.318689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457199:3361], now have 1 active actors on pipe
2025-11-27T14:52:23.318759Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.318769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.318803Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|30d57e4-2cd8472-2cc48f08-b9611bc1_4 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.318816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.318820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.318824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.318826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.318830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.318837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.318840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.318843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.318856Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.318871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.318916Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|30d57e4-2cd8472-2cc48f08-b9611bc1_4"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|30d57e4-2cd8472-2cc48f08-b9611bc1_4"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 4
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.319544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457199:3361] destroyed
2025-11-27T14:52:23.321103Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.321549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457202:3362], now have 1 active actors on pipe
2025-11-27T14:52:23.321877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.321890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.321907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 4 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.321915Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.321924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.321927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.321934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.321936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.321938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.321956Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 0
2025-11-27T14:52:23.321986Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 4 partNo 0 FormedBlobsCount 0 NewHead: Offset 4 PartNo 0 PackedSize 262214 count 1 nextOffset 5 batches 1
2025-11-27T14:52:23.321994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.321996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.321998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.322075Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 4,1 HeadOffset 4 endOffset 4 curOffset 5 d0000000000_00000000000000000004_00000_0000000001_00000? size 262204 WTime 1764255143321
2025-11-27T14:52:23.322105Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.322116Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 4 partNo 0 count 1 size 262204
2025-11-27T14:52:23.324094Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 4 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.324114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.324134Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.324135Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.324141Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.324159Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 4, partNo: 0, Offset: 4 is stored on disk
2025-11-27T14:52:23.324186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.324191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.324192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.324193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.324194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.324199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.324206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.324277Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.324496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457202:3362] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 4
    Offset: 4
    WriteTimestampMS: 1764255143321
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.325065Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.325384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457206:3364], now have 1 active actors on pipe
2025-11-27T14:52:23.325464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.325476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.325501Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|8fcbbd3d-c1a92552-f72c335-dafb8221_5 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.325507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.325509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.325512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.325514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.325517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.325524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.325526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.325528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.325537Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.325545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.325582Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.325730Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457206:3364] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|8fcbbd3d-c1a92552-f72c335-dafb8221_5"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|8fcbbd3d-c1a92552-f72c335-dafb8221_5"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 5
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.327520Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.327836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457209:3365], now have 1 active actors on pipe
2025-11-27T14:52:23.328184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.328194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.328210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 5 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.328220Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.328227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.328232Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.328239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.328240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.328242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.328258Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 0
2025-11-27T14:52:23.328289Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 5 partNo 0 FormedBlobsCount 0 NewHead: Offset 5 PartNo 0 PackedSize 262214 count 1 nextOffset 6 batches 1
2025-11-27T14:52:23.328297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.328299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.328300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.328377Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 5,1 HeadOffset 5 endOffset 5 curOffset 6 d0000000000_00000000000000000005_00000_0000000001_00000? size 262204 WTime 1764255143328
2025-11-27T14:52:23.328406Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.328417Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 5 partNo 0 count 1 size 262204
2025-11-27T14:52:23.330299Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 5 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.330320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.330337Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.330340Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.330346Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.330354Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 5, partNo: 0, Offset: 5 is stored on disk
2025-11-27T14:52:23.330383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.330389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.330391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.330393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.330395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.330401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.330411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.330455Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.330715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457209:3365] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 5
    Offset: 5
    WriteTimestampMS: 1764255143327
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.331255Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.331593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457213:3367], now have 1 active actors on pipe
2025-11-27T14:52:23.331655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.331665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.331697Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|884b37a7-ce7568d8-e8ddc96-3899cecc_6 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.331709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.331712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.331715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.331717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.331720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.331726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.331729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.331731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.331741Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.331752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.331802Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|884b37a7-ce7568d8-e8ddc96-3899cecc_6"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
2025-11-27T14:52:23.332080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457213:3367] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|884b37a7-ce7568d8-e8ddc96-3899cecc_6"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 6
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.333568Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.333940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457218:3369], now have 1 active actors on pipe
2025-11-27T14:52:23.334285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.334303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.334324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 6 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.334339Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.334353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.334361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.334370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.334376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.334380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.334404Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 0
2025-11-27T14:52:23.334498Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 6 partNo 0 FormedBlobsCount 0 NewHead: Offset 6 PartNo 0 PackedSize 262214 count 1 nextOffset 7 batches 1
2025-11-27T14:52:23.334511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.334519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.334522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.334643Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 6,1 HeadOffset 6 endOffset 6 curOffset 7 d0000000000_00000000000000000006_00000_0000000001_00000? size 262204 WTime 1764255143334
2025-11-27T14:52:23.334686Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.334703Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 6 partNo 0 count 1 size 262204
2025-11-27T14:52:23.336762Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 6 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.336787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.336808Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.336816Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.336823Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.336833Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 6, partNo: 0, Offset: 6 is stored on disk
2025-11-27T14:52:23.336864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.336871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.336873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.336874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.336876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.336884Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.336893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.336981Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.337266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457218:3369] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 6
    Offset: 6
    WriteTimestampMS: 1764255143333
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.337814Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.338179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457224:3372], now have 1 active actors on pipe
2025-11-27T14:52:23.338304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.338313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.338344Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9e279158-990bf127-178891d-927b1973_7 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.338356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.338358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.338362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.338365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.338368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.338374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.338380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.338383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.338393Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.338407Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.338460Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.338657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457224:3372] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9e279158-990bf127-178891d-927b1973_7"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9e279158-990bf127-178891d-927b1973_7"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 7
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.340368Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.340744Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457227:3373], now have 1 active actors on pipe
2025-11-27T14:52:23.341087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.341101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.341122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 7 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.341139Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.341149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.341158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.341167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.341173Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.341178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.341203Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 0
2025-11-27T14:52:23.341240Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 7 partNo 0 FormedBlobsCount 0 NewHead: Offset 7 PartNo 0 PackedSize 262214 count 1 nextOffset 8 batches 1
2025-11-27T14:52:23.341252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.341255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.341257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.341376Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 7,1 HeadOffset 7 endOffset 7 curOffset 8 d0000000000_00000000000000000007_00000_0000000001_00000? size 262204 WTime 1764255143341
2025-11-27T14:52:23.341421Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.341437Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 7 partNo 0 count 1 size 262204
2025-11-27T14:52:23.343590Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 7 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.343613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.343636Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.343637Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.343644Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.343653Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 7, partNo: 0, Offset: 7 is stored on disk
2025-11-27T14:52:23.343684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.343691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.343693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.343695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.343697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.343705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.343714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.343779Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.344043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457227:3373] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 7
    Offset: 7
    WriteTimestampMS: 1764255143341
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.344604Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.344940Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457231:3375], now have 1 active actors on pipe
2025-11-27T14:52:23.344990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.345000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.345033Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|6ed7133c-cb6e2a33-e9b1d450-b633a40_8 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.345044Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.345047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.345051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.345052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.345055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.345063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.345065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.345068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.345079Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.345093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.345135Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|6ed7133c-cb6e2a33-e9b1d450-b633a40_8"
    Status: Active
    SeqNo: 0
  }
}
2025-11-27T14:52:23.345306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457231:3375] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|6ed7133c-cb6e2a33-e9b1d450-b633a40_8"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 8
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.347042Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.347473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457234:3376], now have 1 active actors on pipe
2025-11-27T14:52:23.347854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.347869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.347888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 8 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.347902Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.347912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.347919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.347927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.347929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.347931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.347953Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 0
2025-11-27T14:52:23.347989Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 8 partNo 0 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 262214 count 1 nextOffset 9 batches 1
2025-11-27T14:52:23.347999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.348001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.348004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.348131Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 8,1 HeadOffset 8 endOffset 8 curOffset 9 d0000000000_00000000000000000008_00000_0000000001_00000? size 262204 WTime 1764255143347
2025-11-27T14:52:23.348175Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.348191Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 8 partNo 0 count 1 size 262204
2025-11-27T14:52:23.350542Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 8 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.350571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.350593Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.350605Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.350606Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.350618Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 8, partNo: 0, Offset: 8 is stored on disk
2025-11-27T14:52:23.350651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.350658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.350660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.350663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.350665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.350672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.350685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.350772Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.351061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457234:3376] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 8
    Offset: 8
    WriteTimestampMS: 1764255143347
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.351671Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.352130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457238:3378], now have 1 active actors on pipe
2025-11-27T14:52:23.352239Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.352251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.352288Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|5d8d3fcc-300090dd-7a5f067f-c260084_9 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.352300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.352303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.352308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.352309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.352311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.352320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.352322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.352325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.352336Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.352353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.352414Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|5d8d3fcc-300090dd-7a5f067f-c260084_9"
    Status: Active
    SeqNo: 0
  }
}
2025-11-27T14:52:23.352607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457238:3378] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|5d8d3fcc-300090dd-7a5f067f-c260084_9"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 9
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.354561Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.354965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457241:3379], now have 1 active actors on pipe
2025-11-27T14:52:23.355350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.355366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.355386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 9 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.355396Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.355413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.355420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.355430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.355453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.355457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.355481Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 0
2025-11-27T14:52:23.355518Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 9 partNo 0 FormedBlobsCount 0 NewHead: Offset 9 PartNo 0 PackedSize 262214 count 1 nextOffset 10 batches 1
2025-11-27T14:52:23.355529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.355531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.355533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.355812Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 9,1 HeadOffset 9 endOffset 9 curOffset 10 d0000000000_00000000000000000009_00000_0000000001_00000? size 262204 WTime 1764255143355
2025-11-27T14:52:23.355864Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.355882Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 9 partNo 0 count 1 size 262204
2025-11-27T14:52:23.358077Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 9 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.358103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.358123Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.358128Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.358132Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.358141Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 9, partNo: 0, Offset: 9 is stored on disk
2025-11-27T14:52:23.358194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.358201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.358203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.358206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.358209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.358216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.358229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.358309Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.358550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457241:3379] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 9
    Offset: 9
    WriteTimestampMS: 1764255143355
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.359122Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.359505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457245:3381], now have 1 active actors on pipe
2025-11-27T14:52:23.359583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.359594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.359625Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|653cb4b8-1288ea2e-cca256e1-f7a55882_10 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.359637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.359639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.359643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.359645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.359647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.359655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.359657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.359658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.359667Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.359675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.359719Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.359881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457245:3381] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|653cb4b8-1288ea2e-cca256e1-f7a55882_10"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|653cb4b8-1288ea2e-cca256e1-f7a55882_10"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 10
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.361282Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.361615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457248:3382], now have 1 active actors on pipe
2025-11-27T14:52:23.361950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.361966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.361986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 10 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.362004Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.362014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.362022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.362030Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.362032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.362036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.362062Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 0
2025-11-27T14:52:23.362098Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 10 partNo 0 FormedBlobsCount 0 NewHead: Offset 10 PartNo 0 PackedSize 262214 count 1 nextOffset 11 batches 1
2025-11-27T14:52:23.362110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.362112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.362115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.362220Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 10,1 HeadOffset 10 endOffset 10 curOffset 11 d0000000000_00000000000000000010_00000_0000000001_00000? size 262204 WTime 1764255143362
2025-11-27T14:52:23.362265Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.362283Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 10 partNo 0 count 1 size 262204
2025-11-27T14:52:23.364290Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 10 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.364316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.364336Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.364336Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.364349Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.364357Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 10, partNo: 0, Offset: 10 is stored on disk
2025-11-27T14:52:23.364389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.364396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.364397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.364400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.364402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.364409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.364418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.364490Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.364749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457248:3382] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 10
    Offset: 10
    WriteTimestampMS: 1764255143361
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.365444Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.365822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457252:3384], now have 1 active actors on pipe
2025-11-27T14:52:23.365907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.365918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.365953Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|2dd0dad6-675f093a-c42364d-ae8324bc_11 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.365965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.365969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.365974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.365978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.365981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.365989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.365993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.365996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.366006Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.366021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.366076Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.366262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457252:3384] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|2dd0dad6-675f093a-c42364d-ae8324bc_11"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|2dd0dad6-675f093a-c42364d-ae8324bc_11"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 11
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.368165Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.368578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457255:3385], now have 1 active actors on pipe
2025-11-27T14:52:23.368943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.368958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.368981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 11 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.368991Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.369008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.369016Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.369029Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.369036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.369039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.369066Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 0
2025-11-27T14:52:23.369104Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 11 partNo 0 FormedBlobsCount 0 NewHead: Offset 11 PartNo 0 PackedSize 262214 count 1 nextOffset 12 batches 1
2025-11-27T14:52:23.369115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.369118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.369121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.369240Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 11,1 HeadOffset 11 endOffset 11 curOffset 12 d0000000000_00000000000000000011_00000_0000000001_00000? size 262204 WTime 1764255143368
2025-11-27T14:52:23.369282Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.369299Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 11 partNo 0 count 1 size 262204
2025-11-27T14:52:23.371600Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 11 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.371629Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.371650Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.371651Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.371660Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.371669Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 11, partNo: 0, Offset: 11 is stored on disk
2025-11-27T14:52:23.371704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.371713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.371716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.371719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.371722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.371730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.371746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.371822Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.372153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457255:3385] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 11
    Offset: 11
    WriteTimestampMS: 1764255143368
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.372865Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.372944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.372955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.372958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.372960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.372962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.373245Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457259:3387], now have 1 active actors on pipe
2025-11-27T14:52:23.373330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.373342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.373376Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|d5c28e3d-28ab6013-c747fd0a-6fde2d26_12 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.373387Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.373390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.373394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.373397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.373400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.373408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.373415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.373418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.373427Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.373443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.373507Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|d5c28e3d-28ab6013-c747fd0a-6fde2d26_12"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
2025-11-27T14:52:23.373763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457259:3387] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|d5c28e3d-28ab6013-c747fd0a-6fde2d26_12"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 12
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.375650Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.376029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457262:3388], now have 1 active actors on pipe
2025-11-27T14:52:23.376408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.376424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.376446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 12 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.376457Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.376470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.376479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.376488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.376490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.376494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.376521Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 0
2025-11-27T14:52:23.376556Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 12 partNo 0 FormedBlobsCount 0 NewHead: Offset 12 PartNo 0 PackedSize 262214 count 1 nextOffset 13 batches 1
2025-11-27T14:52:23.376568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.376571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.376574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.376690Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 12,1 HeadOffset 12 endOffset 12 curOffset 13 d0000000000_00000000000000000012_00000_0000000001_00000? size 262204 WTime 1764255143376
2025-11-27T14:52:23.376737Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.376755Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 12 partNo 0 count 1 size 262204
2025-11-27T14:52:23.379040Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 12 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.379066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.379086Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.379090Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.379095Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.379104Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 12, partNo: 0, Offset: 12 is stored on disk
2025-11-27T14:52:23.379134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.379144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.379146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.379149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.379151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.379158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.379169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.379235Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 12
    Offset: 12
    WriteTimestampMS: 1764255143376
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-27T14:52:23.379538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457262:3388] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.380159Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.380553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457266:3390], now have 1 active actors on pipe
2025-11-27T14:52:23.380672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.380684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.380713Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b9222f14-5183b8d4-b19db5cd-4373e886_13 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.380725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.380728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.380733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.380741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.380744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.380753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.380756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.380758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.380769Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.380784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.380842Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.381060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457266:3390] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b9222f14-5183b8d4-b19db5cd-4373e886_13"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b9222f14-5183b8d4-b19db5cd-4373e886_13"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 13
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.383011Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.383448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457269:3391], now have 1 active actors on pipe
2025-11-27T14:52:23.383879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.383896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.383921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 13 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.383938Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.383955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.383963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.383974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.383976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.383981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.384007Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 0
2025-11-27T14:52:23.384071Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 13 partNo 0 FormedBlobsCount 0 NewHead: Offset 13 PartNo 0 PackedSize 262214 count 1 nextOffset 14 batches 1
2025-11-27T14:52:23.384083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.384086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.384089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.384224Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 13,1 HeadOffset 13 endOffset 13 curOffset 14 d0000000000_00000000000000000013_00000_0000000001_00000? size 262204 WTime 1764255143384
2025-11-27T14:52:23.384266Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.384283Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 13 partNo 0 count 1 size 262204
2025-11-27T14:52:23.386716Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 13 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.386743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.386767Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.386770Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.386780Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.386789Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 13, partNo: 0, Offset: 13 is stored on disk
2025-11-27T14:52:23.386827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.386835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.386838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.386841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.386843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.386851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.386864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.386958Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.387336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457269:3391] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 13
    Offset: 13
    WriteTimestampMS: 1764255143383
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.388109Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.388538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457273:3393], now have 1 active actors on pipe
2025-11-27T14:52:23.388612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.388624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.388660Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|5f5b972b-749d1fe2-431b080c-f87c2333_14 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.388674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.388678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.388685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.388688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.388691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.388699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.388708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.388710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.388724Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.388741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.388790Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.389012Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457273:3393] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|5f5b972b-749d1fe2-431b080c-f87c2333_14"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|5f5b972b-749d1fe2-431b080c-f87c2333_14"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 14
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.391146Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.391581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457276:3394], now have 1 active actors on pipe
2025-11-27T14:52:23.391954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.391970Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.391993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 14 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.392008Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.392025Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.392033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.392042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.392050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.392055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.392091Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 0
2025-11-27T14:52:23.392128Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 14 partNo 0 FormedBlobsCount 0 NewHead: Offset 14 PartNo 0 PackedSize 262214 count 1 nextOffset 15 batches 1
2025-11-27T14:52:23.392140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.392142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.392145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.392277Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 14,1 HeadOffset 14 endOffset 14 curOffset 15 d0000000000_00000000000000000014_00000_0000000001_00000? size 262204 WTime 1764255143391
2025-11-27T14:52:23.392330Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.392347Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 14 partNo 0 count 1 size 262204
2025-11-27T14:52:23.394467Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 14 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.394490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.394511Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.394513Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.394520Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.394528Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 14, partNo: 0, Offset: 14 is stored on disk
2025-11-27T14:52:23.394556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.394564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.394567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.394569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.394571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.394580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.394590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.394688Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.394981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457276:3394] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 14
    Offset: 14
    WriteTimestampMS: 1764255143391
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.395699Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.396113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457280:3396], now have 1 active actors on pipe
2025-11-27T14:52:23.396186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.396197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.396227Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|2454b89c-7692739-137afa3d-89ed8320_15 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.396239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.396242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.396247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.396249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.396251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.396260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.396263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.396266Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.396277Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.396288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.396344Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.396539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457280:3396] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|2454b89c-7692739-137afa3d-89ed8320_15"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|2454b89c-7692739-137afa3d-89ed8320_15"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 15
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.398658Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.399068Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457283:3397], now have 1 active actors on pipe
2025-11-27T14:52:23.399457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.399476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.399498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 15 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.399509Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.399520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.399523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.399532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.399540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.399543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.399569Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 0
2025-11-27T14:52:23.399640Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 15 partNo 0 FormedBlobsCount 0 NewHead: Offset 15 PartNo 0 PackedSize 262214 count 1 nextOffset 16 batches 1
2025-11-27T14:52:23.399652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.399655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.399658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.399779Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 15,1 HeadOffset 15 endOffset 15 curOffset 16 d0000000000_00000000000000000015_00000_0000000001_00000? size 262204 WTime 1764255143399
2025-11-27T14:52:23.399822Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.399839Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 15 partNo 0 count 1 size 262204
2025-11-27T14:52:23.401853Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 15 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.401876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.401897Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.401896Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.401909Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.401918Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 15, partNo: 0, Offset: 15 is stored on disk
2025-11-27T14:52:23.401956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.401963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.401965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.401967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.401969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.401975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.401987Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.402056Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.402374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457283:3397] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 15
    Offset: 15
    WriteTimestampMS: 1764255143399
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.402994Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.403365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457287:3399], now have 1 active actors on pipe
2025-11-27T14:52:23.403461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.403472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.403504Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|7098f4c-bed9cb80-6f1b2ebf-c77473cc_16 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.403519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.403523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.403528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.403530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.403533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.403540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.403544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.403546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.403559Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.403568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.403624Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.403836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457287:3399] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|7098f4c-bed9cb80-6f1b2ebf-c77473cc_16"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|7098f4c-bed9cb80-6f1b2ebf-c77473cc_16"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 16
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.405660Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.406030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457290:3400], now have 1 active actors on pipe
2025-11-27T14:52:23.406696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.406715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.406737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 16 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.406747Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.406768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.406776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.406786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.406791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.406795Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.406821Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 0
2025-11-27T14:52:23.406857Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 16 partNo 0 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 262214 count 1 nextOffset 17 batches 1
2025-11-27T14:52:23.406871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.406874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.406876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.406994Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 16,1 HeadOffset 16 endOffset 16 curOffset 17 d0000000000_00000000000000000016_00000_0000000001_00000? size 262204 WTime 1764255143406
2025-11-27T14:52:23.407044Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.407061Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 16 partNo 0 count 1 size 262204
2025-11-27T14:52:23.409420Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 16 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.409451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.409469Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.409474Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.409483Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.409493Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 16, partNo: 0, Offset: 16 is stored on disk
2025-11-27T14:52:23.409524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.409533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.409536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.409538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.409541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.409550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.409563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.409669Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.410006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457290:3400] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 16
    Offset: 16
    WriteTimestampMS: 1764255143406
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.410708Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.411111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457294:3402], now have 1 active actors on pipe
2025-11-27T14:52:23.411216Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.411227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.411261Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|2d3c526a-2914f264-4624fc1f-a49d44ad_17 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.411273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.411277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.411287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.411290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.411293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.411302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.411309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.411312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.411324Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.411341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.411411Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.411650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457294:3402] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|2d3c526a-2914f264-4624fc1f-a49d44ad_17"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|2d3c526a-2914f264-4624fc1f-a49d44ad_17"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 17
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.413565Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.413904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457297:3403], now have 1 active actors on pipe
2025-11-27T14:52:23.414256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.414269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.414290Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 17 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.414300Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.414315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.414322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.414332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.414338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.414343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.414379Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 0
2025-11-27T14:52:23.414434Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 17 partNo 0 FormedBlobsCount 0 NewHead: Offset 17 PartNo 0 PackedSize 262214 count 1 nextOffset 18 batches 1
2025-11-27T14:52:23.414446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.414449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.414452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.414568Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 17,1 HeadOffset 17 endOffset 17 curOffset 18 d0000000000_00000000000000000017_00000_0000000001_00000? size 262204 WTime 1764255143414
2025-11-27T14:52:23.414609Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.414626Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 17 partNo 0 count 1 size 262204
2025-11-27T14:52:23.416819Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 17 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.416847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.416868Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.416881Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.416891Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 17, partNo: 0, Offset: 17 is stored on disk
2025-11-27T14:52:23.416891Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.416923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.416931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.416933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.416935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.416937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.416945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.416957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.417062Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.417335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457297:3403] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 17
    Offset: 17
    WriteTimestampMS: 1764255143413
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.417981Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.418330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457301:3405], now have 1 active actors on pipe
2025-11-27T14:52:23.418409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.418419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.418451Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|c3778834-fbdcdc05-807c9921-5066d9f2_18 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.418463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.418467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.418472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.418479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.418482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.418492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.418498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.418501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.418511Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.418525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.418591Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.418795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457301:3405] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|c3778834-fbdcdc05-807c9921-5066d9f2_18"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|c3778834-fbdcdc05-807c9921-5066d9f2_18"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 18
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.420754Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.421096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457304:3406], now have 1 active actors on pipe
2025-11-27T14:52:23.421417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.421433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.421454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 18 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.421465Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.421477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.421486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.421496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.421504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.421508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.421532Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 0
2025-11-27T14:52:23.421568Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 18 partNo 0 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 262214 count 1 nextOffset 19 batches 1
2025-11-27T14:52:23.421580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.421582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.421584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.421694Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 18,1 HeadOffset 18 endOffset 18 curOffset 19 d0000000000_00000000000000000018_00000_0000000001_00000? size 262204 WTime 1764255143421
2025-11-27T14:52:23.421739Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.421756Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 18 partNo 0 count 1 size 262204
2025-11-27T14:52:23.423805Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 18 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.423831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.423843Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.423852Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.423862Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.423871Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 18, partNo: 0, Offset: 18 is stored on disk
2025-11-27T14:52:23.423901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.423909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.423911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.423914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.423917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.423924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.423934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.424012Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.424317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457304:3406] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 18
    Offset: 18
    WriteTimestampMS: 1764255143421
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.424927Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.425264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457308:3408], now have 1 active actors on pipe
2025-11-27T14:52:23.425367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.425378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.425411Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|42a81d40-eac2febc-2222f2ee-12dee29_19 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.425422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.425425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.425429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.425431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.425434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.425442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.425444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.425446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.425457Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.425468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.425527Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.425760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457308:3408] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|42a81d40-eac2febc-2222f2ee-12dee29_19"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|42a81d40-eac2febc-2222f2ee-12dee29_19"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 19
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.427524Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.427919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457311:3409], now have 1 active actors on pipe
2025-11-27T14:52:23.428291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.428307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.428329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 19 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.428339Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.428353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.428356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.428366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.428373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.428377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.428403Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 0
2025-11-27T14:52:23.428442Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 19 partNo 0 FormedBlobsCount 0 NewHead: Offset 19 PartNo 0 PackedSize 262214 count 1 nextOffset 20 batches 1
2025-11-27T14:52:23.428454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.428456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.428459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.428581Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 19,1 HeadOffset 19 endOffset 19 curOffset 20 d0000000000_00000000000000000019_00000_0000000001_00000? size 262204 WTime 1764255143428
2025-11-27T14:52:23.428622Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.428640Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 19 partNo 0 count 1 size 262204
2025-11-27T14:52:23.430710Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 19 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.430737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.430766Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.430765Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.430779Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.430789Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 19, partNo: 0, Offset: 19 is stored on disk
2025-11-27T14:52:23.430821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.430828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.430830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.430832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.430834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.430842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.430852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.430945Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.431228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457311:3409] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 19
    Offset: 19
    WriteTimestampMS: 1764255143428
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.431929Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.432321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457315:3411], now have 1 active actors on pipe
2025-11-27T14:52:23.432418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.432430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.432465Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f1810156-58fe2d6e-fbfdb91b-f6e16ec_20 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.432478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.432481Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.432484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.432486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.432489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.432496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.432499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.432502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.432512Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.432523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.432575Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.432775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457315:3411] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f1810156-58fe2d6e-fbfdb91b-f6e16ec_20"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f1810156-58fe2d6e-fbfdb91b-f6e16ec_20"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 20
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.434643Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.435034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457318:3412], now have 1 active actors on pipe
2025-11-27T14:52:23.435494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.435511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.435536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 20 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.435553Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.435571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.435580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.435590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.435593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.435597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.435626Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 0
2025-11-27T14:52:23.435662Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 20 partNo 0 FormedBlobsCount 0 NewHead: Offset 20 PartNo 0 PackedSize 262214 count 1 nextOffset 21 batches 1
2025-11-27T14:52:23.435673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.435676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.435679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.435807Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 20,1 HeadOffset 20 endOffset 20 curOffset 21 d0000000000_00000000000000000020_00000_0000000001_00000? size 262204 WTime 1764255143435
2025-11-27T14:52:23.435864Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.435882Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 20 partNo 0 count 1 size 262204
2025-11-27T14:52:23.438158Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 20 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.438187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.438204Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.438208Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.438218Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.438228Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 20, partNo: 0, Offset: 20 is stored on disk
2025-11-27T14:52:23.438260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.438268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.438270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.438272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.438274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.438281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.438292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.438379Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.438670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457318:3412] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 20
    Offset: 20
    WriteTimestampMS: 1764255143435
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 1
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.439319Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.439741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457325:3414], now have 1 active actors on pipe
2025-11-27T14:52:23.439842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.439855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.439891Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|4b0ff22-b6c0b1af-22e42635-72127e18_21 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.439904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.439912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.439917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.439919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.439922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.439931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.439934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.439938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.439966Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.439983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.440035Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.440252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457325:3414] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|4b0ff22-b6c0b1af-22e42635-72127e18_21"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|4b0ff22-b6c0b1af-22e42635-72127e18_21"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 21
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.442317Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.442689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457328:3415], now have 1 active actors on pipe
2025-11-27T14:52:23.443057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.443071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.443093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 21 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.443109Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.443124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.443132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.443144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.443146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.443150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.443175Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 0
2025-11-27T14:52:23.443211Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 21 partNo 0 FormedBlobsCount 0 NewHead: Offset 21 PartNo 0 PackedSize 262214 count 1 nextOffset 22 batches 1
2025-11-27T14:52:23.443222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.443224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.443228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.443362Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 21,1 HeadOffset 21 endOffset 21 curOffset 22 d0000000000_00000000000000000021_00000_0000000001_00000? size 262204 WTime 1764255143441
2025-11-27T14:52:23.443409Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.443426Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 21 partNo 0 count 1 size 262204
2025-11-27T14:52:23.445626Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 21 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.445658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.445687Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.445703Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:490: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-11-27T14:52:23.445729Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.445742Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.445752Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 21, partNo: 0, Offset: 21 is stored on disk
2025-11-27T14:52:23.445787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.445794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.445796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.445799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.445801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.445809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.445820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.445928Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.446233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457328:3415] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 21
    Offset: 21
    WriteTimestampMS: 1764255143441
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.447077Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.447741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457333:3418], now have 1 active actors on pipe
2025-11-27T14:52:23.447863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.447875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.447908Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|2eae5446-699c07c-78335a36-11ac59ba_22 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.447922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.447927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.447932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.447934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.447937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.447946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.447949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.447951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.447964Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.447980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.448096Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.448333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457333:3418] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|2eae5446-699c07c-78335a36-11ac59ba_22"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|2eae5446-699c07c-78335a36-11ac59ba_22"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 22
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.450507Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.450891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457336:3419], now have 1 active actors on pipe
2025-11-27T14:52:23.454140Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 22
    Offset: 22
    WriteTimestampMS: 1764255143451
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.451270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.451285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.451308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 22 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.451319Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.451332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.451336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.451365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.451367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.451370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.451408Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 0
2025-11-27T14:52:23.451458Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 22 partNo 0 FormedBlobsCount 0 NewHead: Offset 22 PartNo 0 PackedSize 262214 count 1 nextOffset 23 batches 1
2025-11-27T14:52:23.451465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.451468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.451471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.451597Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 22,1 HeadOffset 22 endOffset 22 curOffset 23 d0000000000_00000000000000000022_00000_0000000001_00000? size 262204 WTime 1764255143451
2025-11-27T14:52:23.451634Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.451645Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 22 partNo 0 count 1 size 262204
2025-11-27T14:52:23.453928Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 22 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.453955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.453975Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.453978Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.453989Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.453998Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 22, partNo: 0, Offset: 22 is stored on disk
2025-11-27T14:52:23.454033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.454036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.454038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.454041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.454043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.454051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.454063Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.454470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457336:3419] destroyed
2025-11-27T14:52:23.455925Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.456369Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457340:3421], now have 1 active actors on pipe
2025-11-27T14:52:23.456485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.456496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.456531Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|4ae18bec-aa20b242-1ef8e280-bf41b93e_23 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.456546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.456549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.456554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.456556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.456560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.456572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.456580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.456583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.456597Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.456614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.456699Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.456952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457340:3421] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|4ae18bec-aa20b242-1ef8e280-bf41b93e_23"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|4ae18bec-aa20b242-1ef8e280-bf41b93e_23"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 23
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.458990Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.459489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457343:3422], now have 1 active actors on pipe
2025-11-27T14:52:23.459911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.459928Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.459952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 23 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.459965Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.459987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.459996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.460007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.460011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.460016Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.460043Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 0
2025-11-27T14:52:23.460094Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 23 partNo 0 FormedBlobsCount 0 NewHead: Offset 23 PartNo 0 PackedSize 262214 count 1 nextOffset 24 batches 1
2025-11-27T14:52:23.460106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.460109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.460127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.460256Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 23,1 HeadOffset 23 endOffset 23 curOffset 24 d0000000000_00000000000000000023_00000_0000000001_00000? size 262204 WTime 1764255143459
2025-11-27T14:52:23.460306Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.460322Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 23 partNo 0 count 1 size 262204
2025-11-27T14:52:23.463184Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 23 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.463210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.463234Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.463236Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.463244Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.463253Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 23, partNo: 0, Offset: 23 is stored on disk
2025-11-27T14:52:23.463295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.463302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.463305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.463307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.463309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.463317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.463329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.463415Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.463725Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457343:3422] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 23
    Offset: 23
    WriteTimestampMS: 1764255143459
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.464423Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.464858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457347:3424], now have 1 active actors on pipe
2025-11-27T14:52:23.464976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.464990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.465036Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|fb9e8b72-6973d621-815a1517-851c33f0_24 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.465050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.465054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.465061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.465068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.465072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.465081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.465084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.465088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.465100Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.465117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.465194Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|fb9e8b72-6973d621-815a1517-851c33f0_24"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
2025-11-27T14:52:23.465493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457347:3424] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|fb9e8b72-6973d621-815a1517-851c33f0_24"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 24
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.467466Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.467948Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457350:3425], now have 1 active actors on pipe
2025-11-27T14:52:23.468346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.468361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.468395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 24 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.468410Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.468427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.468435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.468445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.468451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.468456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.468482Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 0
2025-11-27T14:52:23.468520Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 24 partNo 0 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 262214 count 1 nextOffset 25 batches 1
2025-11-27T14:52:23.468534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.468541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.468544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.468677Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 24,1 HeadOffset 24 endOffset 24 curOffset 25 d0000000000_00000000000000000024_00000_0000000001_00000? size 262204 WTime 1764255143468
2025-11-27T14:52:23.468723Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.468735Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 24 partNo 0 count 1 size 262204
2025-11-27T14:52:23.470964Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 24 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.470989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.471011Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.471011Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.471021Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.471030Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 24, partNo: 0, Offset: 24 is stored on disk
2025-11-27T14:52:23.471064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.471069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.471071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.471074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.471076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.471084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.471095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.471306Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 24
    Offset: 24
    WriteTimestampMS: 1764255143468
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
2025-11-27T14:52:23.471707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457350:3425] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.472266Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.472676Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457354:3427], now have 1 active actors on pipe
2025-11-27T14:52:23.472758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.472769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.472799Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|80d7e349-e08bcef2-71640725-2bde67ec_25 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.472812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.472815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.472819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.472826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.472829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.472838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.472840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.472843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.472854Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.472864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.472973Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.473195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457354:3427] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|80d7e349-e08bcef2-71640725-2bde67ec_25"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
2025-11-27T14:52:23.473293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.473301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.473303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.473305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.473307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|80d7e349-e08bcef2-71640725-2bde67ec_25"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 25
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.475168Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.475607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457357:3428], now have 1 active actors on pipe
2025-11-27T14:52:23.475981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.475999Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.476023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 25 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.476042Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.476068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.476077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.476088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.476094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.476098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.476124Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 0
2025-11-27T14:52:23.476160Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 25 partNo 0 FormedBlobsCount 0 NewHead: Offset 25 PartNo 0 PackedSize 262214 count 1 nextOffset 26 batches 1
2025-11-27T14:52:23.476171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.476174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.476176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.476302Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 25,1 HeadOffset 25 endOffset 25 curOffset 26 d0000000000_00000000000000000025_00000_0000000001_00000? size 262204 WTime 1764255143475
2025-11-27T14:52:23.476357Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.476374Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 25 partNo 0 count 1 size 262204
2025-11-27T14:52:23.478627Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 25 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.478654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.478676Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.478677Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.478691Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.478702Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 25, partNo: 0, Offset: 25 is stored on disk
2025-11-27T14:52:23.478733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.478742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.478746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.478749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.478751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.478759Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.478769Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.478850Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.479131Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457357:3428] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 25
    Offset: 25
    WriteTimestampMS: 1764255143475
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.479808Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.480196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457361:3430], now have 1 active actors on pipe
2025-11-27T14:52:23.480279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.480291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.480326Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|77f4cbc-93bf15c8-663550f-7ac5fa0e_26 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.480339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.480342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.480346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.480348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.480351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.480361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.480363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.480367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.480377Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.480391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.480442Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.480635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457361:3430] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|77f4cbc-93bf15c8-663550f-7ac5fa0e_26"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|77f4cbc-93bf15c8-663550f-7ac5fa0e_26"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 26
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.482593Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.482910Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457365:3432], now have 1 active actors on pipe
2025-11-27T14:52:23.483253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.483269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.483291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 26 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.483301Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.483319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.483322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.483333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.483340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.483345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.483371Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 0
2025-11-27T14:52:23.483410Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 26 partNo 0 FormedBlobsCount 0 NewHead: Offset 26 PartNo 0 PackedSize 262214 count 1 nextOffset 27 batches 1
2025-11-27T14:52:23.483423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.483426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.483430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.483568Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 26,1 HeadOffset 26 endOffset 26 curOffset 27 d0000000000_00000000000000000026_00000_0000000001_00000? size 262204 WTime 1764255143483
2025-11-27T14:52:23.483613Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.483630Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 26 partNo 0 count 1 size 262204
2025-11-27T14:52:23.485637Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 26 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.485677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.485703Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.485705Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.485715Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.485725Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 26, partNo: 0, Offset: 26 is stored on disk
2025-11-27T14:52:23.485753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.485761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.485763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.485765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.485768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.485775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.485785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.485870Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.486125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457365:3432] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 26
    Offset: 26
    WriteTimestampMS: 1764255143482
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.486714Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.487020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457369:3434], now have 1 active actors on pipe
2025-11-27T14:52:23.487106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.487119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.487150Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|4a2267e0-832f252c-902bfd98-5a3975db_27 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.487163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.487166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.487169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.487171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.487175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.487182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.487185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.487188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.487196Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.487206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.487250Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.487430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457369:3434] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|4a2267e0-832f252c-902bfd98-5a3975db_27"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|4a2267e0-832f252c-902bfd98-5a3975db_27"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 27
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.489466Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.489823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457372:3435], now have 1 active actors on pipe
2025-11-27T14:52:23.490179Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.490199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.490222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 27 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.490240Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.490252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.490275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.490286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.490288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.490292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.490316Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 0
2025-11-27T14:52:23.490350Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 27 partNo 0 FormedBlobsCount 0 NewHead: Offset 27 PartNo 0 PackedSize 262214 count 1 nextOffset 28 batches 1
2025-11-27T14:52:23.490380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.490383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.490385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.490499Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 27,1 HeadOffset 27 endOffset 27 curOffset 28 d0000000000_00000000000000000027_00000_0000000001_00000? size 262204 WTime 1764255143490
2025-11-27T14:52:23.490543Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.490560Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 27 partNo 0 count 1 size 262204
2025-11-27T14:52:23.492979Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 27 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.493006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.493028Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.493028Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.493037Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.493047Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 27, partNo: 0, Offset: 27 is stored on disk
2025-11-27T14:52:23.493083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.493092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.493095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.493097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.493099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.493106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.493117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.493240Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.493632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457372:3435] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 27
    Offset: 27
    WriteTimestampMS: 1764255143490
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.494501Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.495031Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457376:3437], now have 1 active actors on pipe
2025-11-27T14:52:23.495152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.495164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.495199Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a3985692-23715657-cfc2dc5e-ec735c3d_28 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.495213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.495219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.495223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.495225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.495229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.495238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.495242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.495245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.495256Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.495279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.495316Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.495617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457376:3437] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a3985692-23715657-cfc2dc5e-ec735c3d_28"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a3985692-23715657-cfc2dc5e-ec735c3d_28"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 28
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.497928Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.498383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457379:3438], now have 1 active actors on pipe
2025-11-27T14:52:23.498750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.498764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.498787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 28 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.498798Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.498819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.498827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.498839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.498846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.498850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.498877Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 0
2025-11-27T14:52:23.498925Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 28 partNo 0 FormedBlobsCount 0 NewHead: Offset 28 PartNo 0 PackedSize 262214 count 1 nextOffset 29 batches 1
2025-11-27T14:52:23.498937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.498941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.498944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.499060Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 28,1 HeadOffset 28 endOffset 28 curOffset 29 d0000000000_00000000000000000028_00000_0000000001_00000? size 262204 WTime 1764255143498
2025-11-27T14:52:23.499122Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.499139Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 28 partNo 0 count 1 size 262204
2025-11-27T14:52:23.501760Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 28 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.501793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.501810Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.501819Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.501828Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.501838Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 28, partNo: 0, Offset: 28 is stored on disk
2025-11-27T14:52:23.501875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.501882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.501884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.501887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.501889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.501897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.501908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.501985Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.502318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457379:3438] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 28
    Offset: 28
    WriteTimestampMS: 1764255143498
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.503456Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.503883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457383:3440], now have 1 active actors on pipe
2025-11-27T14:52:23.503983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.503996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.504034Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|fa05757a-6955edfe-b196ca9-ac8bae16_29 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.504049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.504052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.504065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.504068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.504071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.504091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.504099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.504102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.504116Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.504133Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.504195Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.504450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457383:3440] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|fa05757a-6955edfe-b196ca9-ac8bae16_29"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|fa05757a-6955edfe-b196ca9-ac8bae16_29"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 29
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.507466Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.507871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457386:3441], now have 1 active actors on pipe
2025-11-27T14:52:23.508250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.508268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.508292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 29 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.508305Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.508320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.508329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.508339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.508345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.508353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.508380Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 0
2025-11-27T14:52:23.508416Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 29 partNo 0 FormedBlobsCount 0 NewHead: Offset 29 PartNo 0 PackedSize 262214 count 1 nextOffset 30 batches 1
2025-11-27T14:52:23.508428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.508431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.508434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.508729Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 29,1 HeadOffset 29 endOffset 29 curOffset 30 d0000000000_00000000000000000029_00000_0000000001_00000? size 262204 WTime 1764255143508
2025-11-27T14:52:23.508776Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.508793Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 29 partNo 0 count 1 size 262204
2025-11-27T14:52:23.510838Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 29 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.510867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.510880Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.510891Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.510908Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.510921Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 29, partNo: 0, Offset: 29 is stored on disk
2025-11-27T14:52:23.510971Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.510981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.510983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.510986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.510988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.510997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.511009Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.511071Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.511360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457386:3441] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 29
    Offset: 29
    WriteTimestampMS: 1764255143508
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.512243Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.512666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457390:3443], now have 1 active actors on pipe
2025-11-27T14:52:23.512766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.512778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.512822Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|e48ff610-141a4ceb-360f8cf4-2ed87827_30 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.512837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.512840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.512845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.512849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.512853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.512868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.512871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.512874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.512885Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.512901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.512958Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.513217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457390:3443] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|e48ff610-141a4ceb-360f8cf4-2ed87827_30"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|e48ff610-141a4ceb-360f8cf4-2ed87827_30"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 30
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.517151Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.517601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457393:3444], now have 1 active actors on pipe
2025-11-27T14:52:23.517964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.517979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.518001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 30 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.518018Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.518037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.518045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.518055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.518062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.518067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.518096Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 0
2025-11-27T14:52:23.518137Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 30 partNo 0 FormedBlobsCount 0 NewHead: Offset 30 PartNo 0 PackedSize 262214 count 1 nextOffset 31 batches 1
2025-11-27T14:52:23.518150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.518159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.518162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.518300Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 30,1 HeadOffset 30 endOffset 30 curOffset 31 d0000000000_00000000000000000030_00000_0000000001_00000? size 262204 WTime 1764255143518
2025-11-27T14:52:23.518344Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.518362Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 30 partNo 0 count 1 size 262204
2025-11-27T14:52:23.520692Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 30 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.520717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.520734Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.520740Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.520755Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.520768Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 30, partNo: 0, Offset: 30 is stored on disk
2025-11-27T14:52:23.520803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.520811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.520813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.520815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.520817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.520825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.520835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.520918Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.521265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457393:3444] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 30
    Offset: 30
    WriteTimestampMS: 1764255143517
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.522431Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.522914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457397:3446], now have 1 active actors on pipe
2025-11-27T14:52:23.522933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.522938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.522970Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f3753d4a-59d2d7b0-e2f0170e-909cb3ed_31 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.522982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.522986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.522991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.522998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.523002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.523010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.523013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.523016Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.523026Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.523034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.523126Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.523399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457397:3446] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f3753d4a-59d2d7b0-e2f0170e-909cb3ed_31"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f3753d4a-59d2d7b0-e2f0170e-909cb3ed_31"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 31
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.525680Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.526101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457400:3447], now have 1 active actors on pipe
2025-11-27T14:52:23.526488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.526507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.526529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 31 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.526540Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.526558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.526563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.526574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.526580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.526584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.526611Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 0
2025-11-27T14:52:23.526650Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 31 partNo 0 FormedBlobsCount 0 NewHead: Offset 31 PartNo 0 PackedSize 262214 count 1 nextOffset 32 batches 1
2025-11-27T14:52:23.526661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.526663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.526666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.526785Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 31,1 HeadOffset 31 endOffset 31 curOffset 32 d0000000000_00000000000000000031_00000_0000000001_00000? size 262204 WTime 1764255143526
2025-11-27T14:52:23.526828Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.526845Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 31 partNo 0 count 1 size 262204
2025-11-27T14:52:23.529180Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 31 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.529206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.529221Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.529228Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.529241Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.529257Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 31, partNo: 0, Offset: 31 is stored on disk
2025-11-27T14:52:23.529293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.529301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.529304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.529306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.529309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.529320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000000_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000001_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000002_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000003_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000004_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000005_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000006_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000007_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000008_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000009_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000010_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529355Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000011_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000012_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000013_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000014_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000015_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529369Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000016_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529371Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000017_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000018_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000019_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000023_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000024_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000025_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529396Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000026_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000027_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000028_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000029_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529406Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000030_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000031_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.529411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 32 keys were taken away. Let's read 8390528 bytes
2025-11-27T14:52:23.529424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.529445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 32 blobs
2025-11-27T14:52:23.529452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000000_00000_0000000001_00000?
2025-11-27T14:52:23.529461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000001_00000_0000000001_00000?
2025-11-27T14:52:23.529478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000002_00000_0000000001_00000?
2025-11-27T14:52:23.529494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000003_00000_0000000001_00000?
2025-11-27T14:52:23.529506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000004_00000_0000000001_00000?
2025-11-27T14:52:23.529513Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.529534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000005_00000_0000000001_00000?
2025-11-27T14:52:23.529548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000006_00000_0000000001_00000?
2025-11-27T14:52:23.529561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000007_00000_0000000001_00000?
2025-11-27T14:52:23.529570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000008_00000_0000000001_00000?
2025-11-27T14:52:23.529613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000009_00000_0000000001_00000?
2025-11-27T14:52:23.529626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000010_00000_0000000001_00000?
2025-11-27T14:52:23.529635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000011_00000_0000000001_00000?
2025-11-27T14:52:23.529648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000012_00000_0000000001_00000?
2025-11-27T14:52:23.529661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000013_00000_0000000001_00000?
2025-11-27T14:52:23.529674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000014_00000_0000000001_00000?
2025-11-27T14:52:23.529688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000015_00000_0000000001_00000?
2025-11-27T14:52:23.529700Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000016_00000_0000000001_00000?
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 31
    Offset: 31
    WriteTimestampMS: 1764255143526
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.530668Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.529788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000017_00000_0000000001_00000?
2025-11-27T14:52:23.529801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000018_00000_0000000001_00000?
2025-11-27T14:52:23.529810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000019_00000_0000000001_00000?
2025-11-27T14:52:23.529818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00000_0000000001_00000?
2025-11-27T14:52:23.529827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00000?
2025-11-27T14:52:23.529835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000001_00000?
2025-11-27T14:52:23.529843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000023_00000_0000000001_00000?
2025-11-27T14:52:23.529851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000024_00000_0000000001_00000?
2025-11-27T14:52:23.529859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000025_00000_0000000001_00000?
2025-11-27T14:52:23.529868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000026_00000_0000000001_00000?
2025-11-27T14:52:23.529877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000027_00000_0000000001_00000?
2025-11-27T14:52:23.529886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000028_00000_0000000001_00000?
2025-11-27T14:52:23.529894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000029_00000_0000000001_00000?
2025-11-27T14:52:23.529902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000030_00000_0000000001_00000?
2025-11-27T14:52:23.529909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000031_00000_0000000001_00000?
2025-11-27T14:52:23.529920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 32 blobs for compaction
2025-11-27T14:52:23.529929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457400:3447] destroyed
2025-11-27T14:52:23.529938Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 1 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.529941Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 1 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.529944Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 2 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.529947Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 3 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.529961Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 4 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.529964Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 5 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.529967Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 6 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.529970Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 7 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.529974Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 8 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.529976Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 9 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.529980Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 10 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.529983Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 11 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.529987Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 12 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.529990Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 13 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.529993Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 14 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.529996Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 15 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.529999Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 16 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.530002Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 17 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.530006Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 18 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.530009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 19 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.530012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 20 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.530015Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 21 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.530019Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 22 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.530022Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 23 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.530026Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 24 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.530029Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 25 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.530032Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 26 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.530035Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 27 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.530039Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 28 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.530042Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 29 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.530044Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 30 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.530048Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 31 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.530065Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 32 blobs are from cache.
2025-11-27T14:52:23.530090Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530097Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530101Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530104Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530108Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-27T14:52:23.530111Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530115Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/32] d0000000000_00000000000000000000_00000_0000000001_00000?
2025-11-27T14:52:23.530118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.530119Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530123Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000000_00000_0000000001_00000?, parameters.CurOffset=0
2025-11-27T14:52:23.530127Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530132Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530135Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530138Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530142Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530145Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530149Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530152Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530155Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530158Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530161Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530165Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530167Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530172Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530174Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530177Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530181Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530184Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530187Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530190Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530193Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530196Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530198Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.530203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 0.0/1
2025-11-27T14:52:23.530211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 0
2025-11-27T14:52:23.530231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 0 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 262214 count 1 nextOffset 1 batches 1
2025-11-27T14:52:23.530235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 0:0 appended
2025-11-27T14:52:23.530240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/32] d0000000000_00000000000000000001_00000_0000000001_00000?
2025-11-27T14:52:23.530242Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.530244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000001_00000_0000000001_00000?, parameters.CurOffset=1
2025-11-27T14:52:23.530316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 1.0/1
2025-11-27T14:52:23.530320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 0
2025-11-27T14:52:23.530746Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 1 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 524387 count 2 nextOffset 2 batches 1
2025-11-27T14:52:23.530750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 1:0 appended
2025-11-27T14:52:23.530755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/32] d0000000000_00000000000000000002_00000_0000000001_00000?
2025-11-27T14:52:23.530758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.530760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000002_00000_0000000001_00000?, parameters.CurOffset=2
2025-11-27T14:52:23.530821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 2.0/1
2025-11-27T14:52:23.530825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 0
2025-11-27T14:52:23.530835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 2 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 786601 count 3 nextOffset 3 batches 2
2025-11-27T14:52:23.530838Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 2:0 appended
2025-11-27T14:52:23.530840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/32] d0000000000_00000000000000000003_00000_0000000001_00000?
2025-11-27T14:52:23.530843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.530845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000003_00000_0000000001_00000?, parameters.CurOffset=3
2025-11-27T14:52:23.530898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 3.0/1
2025-11-27T14:52:23.530902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 0
2025-11-27T14:52:23.531419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 3 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1048774 count 4 nextOffset 4 batches 2
2025-11-27T14:52:23.531425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 3:0 appended
2025-11-27T14:52:23.531430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/32] d0000000000_00000000000000000004_00000_0000000001_00000?
2025-11-27T14:52:23.531432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.531453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000004_00000_0000000001_00000?, parameters.CurOffset=4
2025-11-27T14:52:23.531517Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 4.0/1
2025-11-27T14:52:23.531522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 0
2025-11-27T14:52:23.531534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 4 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1310988 count 5 nextOffset 5 batches 3
2025-11-27T14:52:23.531538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 4:0 appended
2025-11-27T14:52:23.531541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/32] d0000000000_00000000000000000005_00000_0000000001_00000?
2025-11-27T14:52:23.531543Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.531545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000005_00000_0000000001_00000?, parameters.CurOffset=5
2025-11-27T14:52:23.531602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 5.0/1
2025-11-27T14:52:23.531605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 0
2025-11-27T14:52:23.532134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 5 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1573161 count 6 nextOffset 6 batches 3
2025-11-27T14:52:23.532138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 5:0 appended
2025-11-27T14:52:23.532144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/32] d0000000000_00000000000000000006_00000_0000000001_00000?
2025-11-27T14:52:23.532147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.532149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000006_00000_0000000001_00000?, parameters.CurOffset=6
2025-11-27T14:52:23.532212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 6.0/1
2025-11-27T14:52:23.532217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 0
2025-11-27T14:52:23.532228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 6 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1835375 count 7 nextOffset 7 batches 4
2025-11-27T14:52:23.532230Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 6:0 appended
2025-11-27T14:52:23.532236Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/32] d0000000000_00000000000000000007_00000_0000000001_00000?
2025-11-27T14:52:23.532237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.532241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000007_00000_0000000001_00000?, parameters.CurOffset=7
2025-11-27T14:52:23.532299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 7.0/1
2025-11-27T14:52:23.532303Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 0
2025-11-27T14:52:23.532815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 7 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2097548 count 8 nextOffset 8 batches 4
2025-11-27T14:52:23.532820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 7:0 appended
2025-11-27T14:52:23.532824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[8/32] d0000000000_00000000000000000008_00000_0000000001_00000?
2025-11-27T14:52:23.532826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.532829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000008_00000_0000000001_00000?, parameters.CurOffset=8
2025-11-27T14:52:23.532883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 8.0/1
2025-11-27T14:52:23.532888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 0
2025-11-27T14:52:23.532900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 8 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2359762 count 9 nextOffset 9 batches 5
2025-11-27T14:52:23.532902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 8:0 appended
2025-11-27T14:52:23.532905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[9/32] d0000000000_00000000000000000009_00000_0000000001_00000?
2025-11-27T14:52:23.532907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.532912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000009_00000_0000000001_00000?, parameters.CurOffset=9
2025-11-27T14:52:23.532968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 9.0/1
2025-11-27T14:52:23.532972Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 0
2025-11-27T14:52:23.534107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 9 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2621935 count 10 nextOffset 10 batches 5
2025-11-27T14:52:23.534124Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 9:0 appended
2025-11-27T14:52:23.534130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[10/32] d0000000000_00000000000000000010_00000_0000000001_00000?
2025-11-27T14:52:23.534132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.534136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000010_00000_0000000001_00000?, parameters.CurOffset=10
2025-11-27T14:52:23.534195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 10.0/1
2025-11-27T14:52:23.534206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 0
2025-11-27T14:52:23.534218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 10 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2884149 count 11 nextOffset 11 batches 6
2025-11-27T14:52:23.534220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 10:0 appended
2025-11-27T14:52:23.534224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[11/32] d0000000000_00000000000000000011_00000_0000000001_00000?
2025-11-27T14:52:23.534226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.534229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000011_00000_0000000001_00000?, parameters.CurOffset=11
2025-11-27T14:52:23.534295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 11.0/1
2025-11-27T14:52:23.534305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 0
2025-11-27T14:52:23.534605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 11 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3146322 count 12 nextOffset 12 batches 6
2025-11-27T14:52:23.534618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 11:0 appended
2025-11-27T14:52:23.534623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[12/32] d0000000000_00000000000000000012_00000_0000000001_00000?
2025-11-27T14:52:23.534625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.534629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000012_00000_0000000001_00000?, parameters.CurOffset=12
2025-11-27T14:52:23.534695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 12.0/1
2025-11-27T14:52:23.534706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 0
2025-11-27T14:52:23.534730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 12 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3408536 count 13 nextOffset 13 batches 7
2025-11-27T14:52:23.534737Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 12:0 appended
2025-11-27T14:52:23.534741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[13/32] d0000000000_00000000000000000013_00000_0000000001_00000?
2025-11-27T14:52:23.534743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.534746Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000013_00000_0000000001_00000?, parameters.CurOffset=13
2025-11-27T14:52:23.534810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 13.0/1
2025-11-27T14:52:23.534819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 0
2025-11-27T14:52:23.535123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 13 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3670709 count 14 nextOffset 14 batches 7
2025-11-27T14:52:23.535135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 13:0 appended
2025-11-27T14:52:23.535142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[14/32] d0000000000_00000000000000000014_00000_0000000001_00000?
2025-11-27T14:52:23.535143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.535146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000014_00000_0000000001_00000?, parameters.CurOffset=14
2025-11-27T14:52:23.535198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 14.0/1
2025-11-27T14:52:23.535208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 0
2025-11-27T14:52:23.535219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 14 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3932923 count 15 nextOffset 15 batches 8
2025-11-27T14:52:23.535221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 14:0 appended
2025-11-27T14:52:23.535224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[15/32] d0000000000_00000000000000000015_00000_0000000001_00000?
2025-11-27T14:52:23.535226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.535229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000015_00000_0000000001_00000?, parameters.CurOffset=15
2025-11-27T14:52:23.535304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 15.0/1
2025-11-27T14:52:23.535314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 0
2025-11-27T14:52:23.535834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 15 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4195096 count 16 nextOffset 16 batches 8
2025-11-27T14:52:23.535846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 15:0 appended
2025-11-27T14:52:23.535851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[16/32] d0000000000_00000000000000000016_00000_0000000001_00000?
2025-11-27T14:52:23.535853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.535855Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000016_00000_0000000001_00000?, parameters.CurOffset=16
2025-11-27T14:52:23.535928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 16.0/1
2025-11-27T14:52:23.535939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 0
2025-11-27T14:52:23.535950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 16 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4457310 count 17 nextOffset 17 batches 9
2025-11-27T14:52:23.535952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 16:0 appended
2025-11-27T14:52:23.535955Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[17/32] d0000000000_00000000000000000017_00000_0000000001_00000?
2025-11-27T14:52:23.535957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.535960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000017_00000_0000000001_00000?, parameters.CurOffset=17
2025-11-27T14:52:23.536014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 17.0/1
2025-11-27T14:52:23.536023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 0
2025-11-27T14:52:23.536546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 17 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4719483 count 18 nextOffset 18 batches 9
2025-11-27T14:52:23.536559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 17:0 appended
2025-11-27T14:52:23.536564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[18/32] d0000000000_00000000000000000018_00000_0000000001_00000?
2025-11-27T14:52:23.536566Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.536569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000018_00000_0000000001_00000?, parameters.CurOffset=18
2025-11-27T14:52:23.536627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 18.0/1
2025-11-27T14:52:23.536640Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 0
2025-11-27T14:52:23.536652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 18 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4981697 count 19 nextOffset 19 batches 10
2025-11-27T14:52:23.536656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 18:0 appended
2025-11-27T14:52:23.536659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[19/32] d0000000000_00000000000000000019_00000_0000000001_00000?
2025-11-27T14:52:23.536661Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.536664Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000019_00000_0000000001_00000?, parameters.CurOffset=19
2025-11-27T14:52:23.536717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 19.0/1
2025-11-27T14:52:23.536726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 0
2025-11-27T14:52:23.537233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 19 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 5243870 count 20 nextOffset 20 batches 10
2025-11-27T14:52:23.537247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 19:0 appended
2025-11-27T14:52:23.537253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[20/32] d0000000000_00000000000000000020_00000_0000000001_00000?
2025-11-27T14:52:23.537255Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.537257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000020_00000_0000000001_00000?, parameters.CurOffset=20
2025-11-27T14:52:23.537311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 20.0/1
2025-11-27T14:52:23.537315Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 0
2025-11-27T14:52:23.537326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 20 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 5506084 count 21 nextOffset 21 batches 11
2025-11-27T14:52:23.537331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 20:0 appended
2025-11-27T14:52:23.537335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[21/32] d0000000000_00000000000000000021_00000_0000000001_00000?
2025-11-27T14:52:23.537336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.537338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000021_00000_0000000001_00000?, parameters.CurOffset=21
2025-11-27T14:52:23.537393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 21.0/1
2025-11-27T14:52:23.537404Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 0
2025-11-27T14:52:23.537895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 21 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 5768257 count 22 nextOffset 22 batches 11
2025-11-27T14:52:23.537909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 21:0 appended
2025-11-27T14:52:23.537914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[22/32] d0000000000_00000000000000000022_00000_0000000001_00000?
2025-11-27T14:52:23.537915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.537918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000022_00000_0000000001_00000?, parameters.CurOffset=22
2025-11-27T14:52:23.537973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 22.0/1
2025-11-27T14:52:23.537983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 0
2025-11-27T14:52:23.537995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 22 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 6030471 count 23 nextOffset 23 batches 12
2025-11-27T14:52:23.537997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 22:0 appended
2025-11-27T14:52:23.538002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[23/32] d0000000000_00000000000000000023_00000_0000000001_00000?
2025-11-27T14:52:23.538004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.538007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000023_00000_0000000001_00000?, parameters.CurOffset=23
2025-11-27T14:52:23.538057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 23.0/1
2025-11-27T14:52:23.538066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 0
2025-11-27T14:52:23.538606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 23 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 6292644 count 24 nextOffset 24 batches 12
2025-11-27T14:52:23.538619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 23:0 appended
2025-11-27T14:52:23.538623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[24/32] d0000000000_00000000000000000024_00000_0000000001_00000?
2025-11-27T14:52:23.538625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.538628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000024_00000_0000000001_00000?, parameters.CurOffset=24
2025-11-27T14:52:23.538686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 24.0/1
2025-11-27T14:52:23.538697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 0
2025-11-27T14:52:23.538707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 24 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 6554858 count 25 nextOffset 25 batches 13
2025-11-27T14:52:23.538713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 24:0 appended
2025-11-27T14:52:23.538716Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[25/32] d0000000000_00000000000000000025_00000_0000000001_00000?
2025-11-27T14:52:23.538718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.538721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000025_00000_0000000001_00000?, parameters.CurOffset=25
2025-11-27T14:52:23.538773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 25.0/1
2025-11-27T14:52:23.538783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 0
2025-11-27T14:52:23.539304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 25 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 6817031 count 26 nextOffset 26 batches 13
2025-11-27T14:52:23.539316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 25:0 appended
2025-11-27T14:52:23.539321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[26/32] d0000000000_00000000000000000026_00000_0000000001_00000?
2025-11-27T14:52:23.539324Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.539327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000026_00000_0000000001_00000?, parameters.CurOffset=26
2025-11-27T14:52:23.539381Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 26.0/1
2025-11-27T14:52:23.539390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 0
2025-11-27T14:52:23.539401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 26 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 7079245 count 27 nextOffset 27 batches 14
2025-11-27T14:52:23.539403Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 26:0 appended
2025-11-27T14:52:23.539407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[27/32] d0000000000_00000000000000000027_00000_0000000001_00000?
2025-11-27T14:52:23.539408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.539411Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000027_00000_0000000001_00000?, parameters.CurOffset=27
2025-11-27T14:52:23.539482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 27.0/1
2025-11-27T14:52:23.539491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 0
2025-11-27T14:52:23.539989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 27 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 7341418 count 28 nextOffset 28 batches 14
2025-11-27T14:52:23.540003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 27:0 appended
2025-11-27T14:52:23.540008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[28/32] d0000000000_00000000000000000028_00000_0000000001_00000?
2025-11-27T14:52:23.540011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.540013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000028_00000_0000000001_00000?, parameters.CurOffset=28
2025-11-27T14:52:23.540080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 28.0/1
2025-11-27T14:52:23.540091Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 0
2025-11-27T14:52:23.540116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 28 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 7603632 count 29 nextOffset 29 batches 15
2025-11-27T14:52:23.540123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 28:0 appended
2025-11-27T14:52:23.540128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[29/32] d0000000000_00000000000000000029_00000_0000000001_00000?
2025-11-27T14:52:23.540129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.540132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000029_00000_0000000001_00000?, parameters.CurOffset=29
2025-11-27T14:52:23.540190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 29.0/1
2025-11-27T14:52:23.540201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 0
2025-11-27T14:52:23.540762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 29 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 7865805 count 30 nextOffset 30 batches 15
2025-11-27T14:52:23.540777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 29:0 appended
2025-11-27T14:52:23.540783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[30/32] d0000000000_00000000000000000030_00000_0000000001_00000?
2025-11-27T14:52:23.540785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.540788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000030_00000_0000000001_00000?, parameters.CurOffset=30
2025-11-27T14:52:23.540847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 30.0/1
2025-11-27T14:52:23.540861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 0
2025-11-27T14:52:23.540874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 30 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 8128019 count 31 nextOffset 31 batches 16
2025-11-27T14:52:23.540882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 30:0 appended
2025-11-27T14:52:23.540886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[31/32] d0000000000_00000000000000000031_00000_0000000001_00000?
2025-11-27T14:52:23.540887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.540890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000031_00000_0000000001_00000?, parameters.CurOffset=31
2025-11-27T14:52:23.540945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 31.0/1
2025-11-27T14:52:23.540954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 0
2025-11-27T14:52:23.548678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 31 partNo 0 result is x0000000000_00000000000000000000_00000_0000000031_00000 size 8128009
2025-11-27T14:52:23.548705Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1104: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000000_00000_0000000031_00000 new key d0000000000_00000000000000000000_00000_0000000031_00000 size 8128009 WTime 1764255143548
2025-11-27T14:52:23.548743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 31 partNo 0 FormedBlobsCount 1 NewHead: Offset 31 PartNo 0 PackedSize 262214 count 1 nextOffset 32 batches 1
2025-11-27T14:52:23.548750Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 31:0 appended
2025-11-27T14:52:23.548870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 31,1 HeadOffset 0 endOffset 0 curOffset 32 d0000000000_00000000000000000031_00000_0000000001_00000| size 262204 WTime 1764255143548
2025-11-27T14:52:23.548962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457404:3449], now have 1 active actors on pipe
2025-11-27T14:52:23.548989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.548996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.549012Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.549026Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 31 size 8128009
2025-11-27T14:52:23.549033Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 31 partNo 0 count 1 size 262204
2025-11-27T14:52:23.549068Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ee2a9dba-fa0c05f1-498edaa3-6d02a301_32 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.549078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.549082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.549088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.549090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.549094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.549104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.549107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.549110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.549126Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.549186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.549501Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.549823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457404:3449] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ee2a9dba-fa0c05f1-498edaa3-6d02a301_32"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ee2a9dba-fa0c05f1-498edaa3-6d02a301_32"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 32
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.552520Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.553176Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457409:3451], now have 1 active actors on pipe
2025-11-27T14:52:23.553714Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.553734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.553773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 32 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.553786Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.553801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.553805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.553817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.553820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.553825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.553852Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 32 partNo 0
2025-11-27T14:52:23.553888Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 32 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 262214 count 1 nextOffset 33 batches 1
2025-11-27T14:52:23.553896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.553899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.553904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.554213Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 32,1 HeadOffset 32 endOffset 32 curOffset 33 d0000000000_00000000000000000032_00000_0000000001_00000? size 262204 WTime 1764255143554
2025-11-27T14:52:23.554264Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.554276Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 32 partNo 0 count 1 size 262204
2025-11-27T14:52:23.565706Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.565214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 0 count 31 size 8128009 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.565234Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 31 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.565259Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 32 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.565271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.565313Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 31 parts 0 suffix '0' size 8128009
2025-11-27T14:52:23.565323Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 0 suffix '124' size 262204
2025-11-27T14:52:23.565333Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 32 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.565413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-27T14:52:23.565417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.565422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.565449Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.565460Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.565472Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 32, partNo: 0, Offset: 32 is stored on disk
2025-11-27T14:52:23.565495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.565498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.565500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.565502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.565505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.565567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.565580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.565604Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.565612Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000000_00000_0000000001_00000?(+) to d0000000000_00000000000000000000_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565619Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000001_00000_0000000001_00000?(+) to d0000000000_00000000000000000001_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565621Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000002_00000_0000000001_00000?(+) to d0000000000_00000000000000000002_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565623Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000003_00000_0000000001_00000?(+) to d0000000000_00000000000000000003_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565627Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000004_00000_0000000001_00000?(+) to d0000000000_00000000000000000004_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565630Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000005_00000_0000000001_00000?(+) to d0000000000_00000000000000000005_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565633Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000006_00000_0000000001_00000?(+) to d0000000000_00000000000000000006_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565635Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000007_00000_0000000001_00000?(+) to d0000000000_00000000000000000007_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565638Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000008_00000_0000000001_00000?(+) to d0000000000_00000000000000000008_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565640Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000009_00000_0000000001_00000?(+) to d0000000000_00000000000000000009_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565643Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000010_00000_0000000001_00000?(+) to d0000000000_00000000000000000010_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565645Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000011_00000_0000000001_00000?(+) to d0000000000_00000000000000000011_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565649Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000012_00000_0000000001_00000?(+) to d0000000000_00000000000000000012_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565652Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000013_00000_0000000001_00000?(+) to d0000000000_00000000000000000013_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565654Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000014_00000_0000000001_00000?(+) to d0000000000_00000000000000000014_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565656Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000015_00000_0000000001_00000?(+) to d0000000000_00000000000000000015_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565659Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000016_00000_0000000001_00000?(+) to d0000000000_00000000000000000016_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565661Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000017_00000_0000000001_00000?(+) to d0000000000_00000000000000000017_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565664Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000018_00000_0000000001_00000?(+) to d0000000000_00000000000000000018_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565666Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000019_00000_0000000001_00000?(+) to d0000000000_00000000000000000019_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565669Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000020_00000_0000000001_00000?(+) to d0000000000_00000000000000000020_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565671Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000021_00000_0000000001_00000?(+) to d0000000000_00000000000000000021_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565673Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000022_00000_0000000001_00000?(+) to d0000000000_00000000000000000022_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565676Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000023_00000_0000000001_00000?(+) to d0000000000_00000000000000000023_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565678Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000024_00000_0000000001_00000?(+) to d0000000000_00000000000000000024_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565681Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000025_00000_0000000001_00000?(+) to d0000000000_00000000000000000025_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565683Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000026_00000_0000000001_00000?(+) to d0000000000_00000000000000000026_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565686Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000027_00000_0000000001_00000?(+) to d0000000000_00000000000000000027_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565688Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000028_00000_0000000001_00000?(+) to d0000000000_00000000000000000028_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565691Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000029_00000_0000000001_00000?(+) to d0000000000_00000000000000000029_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565693Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000030_00000_0000000001_00000?(+) to d0000000000_00000000000000000030_00000_0000000001_00000?(+)
2025-11-27T14:52:23.565695Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000031_00000_0000000001_00000?(+) to d0000000000_00000000000000000031_00000_0000000001_00000?(+)
2025-11-27T14:52:23.566154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457409:3451] destroyed
2025-11-27T14:52:23.567061Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 0 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567073Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 1 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567079Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 2 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567082Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 3 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 4 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567090Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 5 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567094Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 6 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567098Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 7 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567103Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 8 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567106Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 9 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567111Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 10 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567114Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 11 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567118Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 12 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567122Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 13 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567126Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 14 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567130Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 15 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567137Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 16 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567141Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 17 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567145Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 18 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567149Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 19 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567153Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 20 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567157Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 21 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567160Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 22 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567165Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 23 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567169Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 24 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567172Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 25 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567177Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 26 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567181Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 27 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567185Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 28 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567189Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 29 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567194Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 30 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567197Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 31 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.567221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.567243Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.567250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
CallPersQueueGRPC response:
2025-11-27T14:52:23.567252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.567254Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.567256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.567257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.567266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.567317Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567327Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567333Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567337Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567342Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567346Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567351Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567355Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 0 suffix '63' size 262204
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 32
    Offset: 32
    WriteTimestampMS: 1764255143551
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 13
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-27T14:52:23.567359Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567363Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567369Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567373Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567378Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567383Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567388Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567392Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567397Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567401Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567408Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567414Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567418Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567422Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567426Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567429Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567433Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 0 suffix '63' size 262204
CallPersQueueGRPC request to localhost:6400
2025-11-27T14:52:23.567451Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567455Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567459Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567464Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 0 suffix '63' size 262204
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.567468Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567472Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.567476Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.568293Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.569488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457414:3454], now have 1 active actors on pipe
2025-11-27T14:52:23.569513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.569520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.569563Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|e5e98d84-1e2aeee4-1e7eb858-e0bfdbb1_33 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.569572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.569575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.569580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.569582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.569586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.569595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.569600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.569602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.569615Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.569629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.569697Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.570106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457414:3454] destroyed
2025-11-27T14:52:23.573660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.573676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.573679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.573682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.573685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|e5e98d84-1e2aeee4-1e7eb858-e0bfdbb1_33"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|e5e98d84-1e2aeee4-1e7eb858-e0bfdbb1_33"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 33
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.576966Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.577538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457418:3456], now have 1 active actors on pipe
2025-11-27T14:52:23.578026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.578040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.578060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 33 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.578072Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.578086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.578094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.578103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.578105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.578109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.578133Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 33 partNo 0
2025-11-27T14:52:23.578162Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 33 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 262214 count 1 nextOffset 34 batches 1
2025-11-27T14:52:23.578169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.578172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.578175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.578329Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 33,1 HeadOffset 33 endOffset 33 curOffset 34 d0000000000_00000000000000000033_00000_0000000001_00000? size 262204 WTime 1764255143578
2025-11-27T14:52:23.578372Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.578385Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 33 partNo 0 count 1 size 262204
2025-11-27T14:52:23.580779Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 33 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.580811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.580830Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 33 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.580841Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.580861Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.580877Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 33, partNo: 0, Offset: 33 is stored on disk
2025-11-27T14:52:23.580918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.580928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.580930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.580933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.580937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.580946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.580959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.581069Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 33
    Offset: 33
    WriteTimestampMS: 1764255143577
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-27T14:52:23.581405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457418:3456] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.582074Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.582583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457422:3458], now have 1 active actors on pipe
2025-11-27T14:52:23.582688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.582698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.582737Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9a5b26c7-cc81353f-57e97203-6ba19de0_34 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.582746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.582753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.582757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.582759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.582764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.582775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.582778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.582781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.582876Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.582795Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.582811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9a5b26c7-cc81353f-57e97203-6ba19de0_34"
    Status: Active
    SeqNo: 0
  }
}
2025-11-27T14:52:23.583094Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457422:3458] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9a5b26c7-cc81353f-57e97203-6ba19de0_34"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 34
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.585202Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.585758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457425:3459], now have 1 active actors on pipe
2025-11-27T14:52:23.586339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.586363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.586390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 34 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.586411Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.586435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.586447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.586459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.586468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.586473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.586505Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 34 partNo 0
2025-11-27T14:52:23.586560Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 34 partNo 0 FormedBlobsCount 0 NewHead: Offset 34 PartNo 0 PackedSize 262214 count 1 nextOffset 35 batches 1
2025-11-27T14:52:23.586575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.586579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.586583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.586731Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 34,1 HeadOffset 34 endOffset 34 curOffset 35 d0000000000_00000000000000000034_00000_0000000001_00000? size 262204 WTime 1764255143586
2025-11-27T14:52:23.586788Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.586807Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 34 partNo 0 count 1 size 262204
2025-11-27T14:52:23.589660Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 34 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.589698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.589719Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 34 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.589729Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.589746Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.589765Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 34, partNo: 0, Offset: 34 is stored on disk
2025-11-27T14:52:23.589812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.589823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.589825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.589828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.589831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.589840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.589855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.589959Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 34
    Offset: 34
    WriteTimestampMS: 1764255143586
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-27T14:52:23.590630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457425:3459] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.591202Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.591763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457429:3461], now have 1 active actors on pipe
2025-11-27T14:52:23.591894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.591911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.591954Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|111a1a85-772f0fb1-dd01924f-292890ac_35 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.591974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.591984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.591989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.591992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.591996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.592013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.592018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.592022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.592039Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.592073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.592153Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|111a1a85-772f0fb1-dd01924f-292890ac_35"
    Status: Active
    SeqNo: 0
  }
}
2025-11-27T14:52:23.592592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457429:3461] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|111a1a85-772f0fb1-dd01924f-292890ac_35"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 35
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.594687Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.595232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457432:3462], now have 1 active actors on pipe
2025-11-27T14:52:23.595732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.595752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.595777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 35 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.595789Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.595806Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.595811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.595821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.595824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.595829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.595864Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 35 partNo 0
2025-11-27T14:52:23.595903Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 35 partNo 0 FormedBlobsCount 0 NewHead: Offset 35 PartNo 0 PackedSize 262214 count 1 nextOffset 36 batches 1
2025-11-27T14:52:23.595910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.595913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.595916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.596078Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 35,1 HeadOffset 35 endOffset 35 curOffset 36 d0000000000_00000000000000000035_00000_0000000001_00000? size 262204 WTime 1764255143595
2025-11-27T14:52:23.596118Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.596131Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 35 partNo 0 count 1 size 262204
2025-11-27T14:52:23.598885Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 35 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.598911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.598937Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.598950Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.598961Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 35, partNo: 0, Offset: 35 is stored on disk
2025-11-27T14:52:23.598997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.599000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.599002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.599007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.599009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.599018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.599032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.599074Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 35 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.599139Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.599517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457432:3462] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 35
    Offset: 35
    WriteTimestampMS: 1764255143595
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.600832Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.601435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457436:3464], now have 1 active actors on pipe
2025-11-27T14:52:23.601461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.601468Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.601512Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|e80f04bf-b180cd70-3ae6c6a8-1affc313_36 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.601522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.601526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.601650Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.601531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.601533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.601537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.601550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.601554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.601557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.601574Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.601589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.601860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457436:3464] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|e80f04bf-b180cd70-3ae6c6a8-1affc313_36"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|e80f04bf-b180cd70-3ae6c6a8-1affc313_36"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 36
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.604170Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.604655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457439:3465], now have 1 active actors on pipe
2025-11-27T14:52:23.605112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.605128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.605154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 36 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.605167Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.605183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.605187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.605199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.605202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.605206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.605252Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 36 partNo 0
2025-11-27T14:52:23.605298Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 36 partNo 0 FormedBlobsCount 0 NewHead: Offset 36 PartNo 0 PackedSize 262214 count 1 nextOffset 37 batches 1
2025-11-27T14:52:23.605308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.605311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.605315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.605470Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 36,1 HeadOffset 36 endOffset 36 curOffset 37 d0000000000_00000000000000000036_00000_0000000001_00000? size 262204 WTime 1764255143605
2025-11-27T14:52:23.605514Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.605528Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 36 partNo 0 count 1 size 262204
2025-11-27T14:52:23.608182Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 36 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.608229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.608258Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.608258Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 36 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.608276Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.608287Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 36, partNo: 0, Offset: 36 is stored on disk
2025-11-27T14:52:23.608322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.608334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.608337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.608339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.608342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.608352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.608364Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.608469Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 36
    Offset: 36
    WriteTimestampMS: 1764255143604
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.609040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457439:3465] destroyed
2025-11-27T14:52:23.609651Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.610161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457443:3467], now have 1 active actors on pipe
2025-11-27T14:52:23.610285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.610301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.610347Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|65782695-d4fcc232-e83b2390-59b34059_37 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.610365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.610369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.610375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.610377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.610381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.610392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.610395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.610398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.610414Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.610427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.610516Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|65782695-d4fcc232-e83b2390-59b34059_37"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
2025-11-27T14:52:23.610985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457443:3467] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|65782695-d4fcc232-e83b2390-59b34059_37"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 37
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.613141Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.613699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457447:3469], now have 1 active actors on pipe
2025-11-27T14:52:23.614135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.614156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.614182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 37 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.614202Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.614226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.614230Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.614248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.614257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.614262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.614295Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 37 partNo 0
2025-11-27T14:52:23.614342Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 37 partNo 0 FormedBlobsCount 0 NewHead: Offset 37 PartNo 0 PackedSize 262214 count 1 nextOffset 38 batches 1
2025-11-27T14:52:23.614357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.614360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.614367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.614524Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 37,1 HeadOffset 37 endOffset 37 curOffset 38 d0000000000_00000000000000000037_00000_0000000001_00000? size 262204 WTime 1764255143614
2025-11-27T14:52:23.614577Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.614596Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 37 partNo 0 count 1 size 262204
2025-11-27T14:52:23.616881Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 37 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.616905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.616929Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.616945Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.616955Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 37, partNo: 0, Offset: 37 is stored on disk
2025-11-27T14:52:23.616993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.617001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.617004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.617007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.617010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.617016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.617027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.617075Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 37 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.617127Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 37
    Offset: 37
    WriteTimestampMS: 1764255143614
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-27T14:52:23.617592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457447:3469] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.618326Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.618896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457451:3471], now have 1 active actors on pipe
2025-11-27T14:52:23.619062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.619076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.619114Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|faefbd31-fc5a4e66-527c7200-6d2c44a4_38 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.619127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.619137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.619142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.619144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.619148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.619158Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.619161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.619168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.619180Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.619196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.619281Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.619570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457451:3471] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|faefbd31-fc5a4e66-527c7200-6d2c44a4_38"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|faefbd31-fc5a4e66-527c7200-6d2c44a4_38"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 38
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.621957Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.622416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457454:3472], now have 1 active actors on pipe
2025-11-27T14:52:23.622925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.622944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.622968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 38 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.622979Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.622999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.623008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.623020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.623022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.623028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.623052Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 38 partNo 0
2025-11-27T14:52:23.623082Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 38 partNo 0 FormedBlobsCount 0 NewHead: Offset 38 PartNo 0 PackedSize 262214 count 1 nextOffset 39 batches 1
2025-11-27T14:52:23.623094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.623097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.623102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.623220Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 38,1 HeadOffset 38 endOffset 38 curOffset 39 d0000000000_00000000000000000038_00000_0000000001_00000? size 262204 WTime 1764255143622
2025-11-27T14:52:23.623265Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.623283Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 38 partNo 0 count 1 size 262204
2025-11-27T14:52:23.626080Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 38 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.626110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.626131Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.626141Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.626149Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 38, partNo: 0, Offset: 38 is stored on disk
2025-11-27T14:52:23.626183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.626186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.626188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.626190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.626195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.626201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.626212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.626269Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 38 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.626342Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.626753Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457454:3472] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 38
    Offset: 38
    WriteTimestampMS: 1764255143622
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.627532Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.628003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457458:3474], now have 1 active actors on pipe
2025-11-27T14:52:23.628128Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.628139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.628176Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|e7c50309-bc4d54f1-3e5e271c-9647dcc8_39 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.628185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.628188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.628192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.628194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.628197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.628204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.628207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.628210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.628220Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.628231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.628297Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|e7c50309-bc4d54f1-3e5e271c-9647dcc8_39"
    Status: Active
    SeqNo: 0
  }
}
2025-11-27T14:52:23.628640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457458:3474] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|e7c50309-bc4d54f1-3e5e271c-9647dcc8_39"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 39
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.631008Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.634077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457461:3475], now have 1 active actors on pipe
2025-11-27T14:52:23.634650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.634668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.634693Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 39 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.634709Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.634728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.634732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.634745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.634747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.634752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.634783Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 39 partNo 0
2025-11-27T14:52:23.634820Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 39 partNo 0 FormedBlobsCount 0 NewHead: Offset 39 PartNo 0 PackedSize 262214 count 1 nextOffset 40 batches 1
2025-11-27T14:52:23.634828Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.634832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.634835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.634975Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 39,1 HeadOffset 39 endOffset 39 curOffset 40 d0000000000_00000000000000000039_00000_0000000001_00000? size 262204 WTime 1764255143634
2025-11-27T14:52:23.635020Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.635032Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 39 partNo 0 count 1 size 262204
2025-11-27T14:52:23.637718Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 39 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.637741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.637763Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.637772Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.637782Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 39, partNo: 0, Offset: 39 is stored on disk
2025-11-27T14:52:23.637813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.637818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.637820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.637823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.637825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.637833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.637846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.637970Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 39 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.638080Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.638533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457461:3475] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 39
    Offset: 39
    WriteTimestampMS: 1764255143634
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.639321Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.640001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457465:3477], now have 1 active actors on pipe
2025-11-27T14:52:23.640158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.640174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.640222Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ef90e698-59bf2c33-a320bd8-25742a58_40 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.640241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.640245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.640252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.640255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.640260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.640272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.640283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.640286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.640307Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.640332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.640417Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ef90e698-59bf2c33-a320bd8-25742a58_40"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
2025-11-27T14:52:23.641049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457465:3477] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ef90e698-59bf2c33-a320bd8-25742a58_40"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 40
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.642754Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.643292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457468:3478], now have 1 active actors on pipe
2025-11-27T14:52:23.643718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.643736Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.643765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 40 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.643777Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.643793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.643798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.643810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.643812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.643827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.643861Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 40 partNo 0
2025-11-27T14:52:23.643904Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 40 partNo 0 FormedBlobsCount 0 NewHead: Offset 40 PartNo 0 PackedSize 262214 count 1 nextOffset 41 batches 1
2025-11-27T14:52:23.643911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.643916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.643919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.644042Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 40,1 HeadOffset 40 endOffset 40 curOffset 41 d0000000000_00000000000000000040_00000_0000000001_00000? size 262204 WTime 1764255143643
2025-11-27T14:52:23.644105Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.644125Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 40 partNo 0 count 1 size 262204
2025-11-27T14:52:23.646497Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 40 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.646525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.646552Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.646570Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.646580Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 40, partNo: 0, Offset: 40 is stored on disk
2025-11-27T14:52:23.646618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.646628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.646631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.646634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.646637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.646645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.646658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.646698Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 40 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.646784Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 40
    Offset: 40
    WriteTimestampMS: 1764255143643
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.647224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457468:3478] destroyed
2025-11-27T14:52:23.647842Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.648287Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457472:3480], now have 1 active actors on pipe
2025-11-27T14:52:23.648392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.648406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.648441Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|31f36432-630ac2a3-722addb3-e64b7b40_41 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.648450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.648454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.648458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.648461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.648465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.648474Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.648562Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.648477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.648480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.648494Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.648507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.648751Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457472:3480] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|31f36432-630ac2a3-722addb3-e64b7b40_41"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|31f36432-630ac2a3-722addb3-e64b7b40_41"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 41
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.654831Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.655323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457476:3482], now have 1 active actors on pipe
2025-11-27T14:52:23.655949Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.655972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.656001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 41 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.656027Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.656053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.656075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.656088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.656097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.656103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.656142Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 41 partNo 0
2025-11-27T14:52:23.656193Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 41 partNo 0 FormedBlobsCount 0 NewHead: Offset 41 PartNo 0 PackedSize 262214 count 1 nextOffset 42 batches 1
2025-11-27T14:52:23.656208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.656212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.656216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.656537Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 41,1 HeadOffset 41 endOffset 41 curOffset 42 d0000000000_00000000000000000041_00000_0000000001_00000? size 262204 WTime 1764255143656
2025-11-27T14:52:23.656629Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.656651Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 41 partNo 0 count 1 size 262204
2025-11-27T14:52:23.659490Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 41 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.659522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.659545Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 41 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.659549Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.659562Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.659572Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 41, partNo: 0, Offset: 41 is stored on disk
2025-11-27T14:52:23.659640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.659650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.659653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.659655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.659658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.659671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.659687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.660172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457476:3482] destroyed
2025-11-27T14:52:23.659813Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 41
    Offset: 41
    WriteTimestampMS: 1764255143656
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.662209Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.662816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457480:3484], now have 1 active actors on pipe
2025-11-27T14:52:23.662955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.662971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.663022Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|7a747a01-a6dc58ef-bb4ebc31-8b8a3420_42 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.663041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.663045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.663051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.663054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.663058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.663073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.663077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.663081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.663100Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.663117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.663196Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.663566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457480:3484] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|7a747a01-a6dc58ef-bb4ebc31-8b8a3420_42"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|7a747a01-a6dc58ef-bb4ebc31-8b8a3420_42"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 42
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.666169Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.666666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457483:3485], now have 1 active actors on pipe
2025-11-27T14:52:23.667151Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.667173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.667199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 42 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.667220Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.667247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.667256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.667268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.667270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.667276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.667310Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 42 partNo 0
2025-11-27T14:52:23.667362Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 42 partNo 0 FormedBlobsCount 0 NewHead: Offset 42 PartNo 0 PackedSize 262214 count 1 nextOffset 43 batches 1
2025-11-27T14:52:23.667376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.667379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.667382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.667556Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 42,1 HeadOffset 42 endOffset 42 curOffset 43 d0000000000_00000000000000000042_00000_0000000001_00000? size 262204 WTime 1764255143667
2025-11-27T14:52:23.667615Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.667635Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 42 partNo 0 count 1 size 262204
2025-11-27T14:52:23.670161Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 42 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.670195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.670220Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 42 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.670222Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.670237Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.670248Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 42, partNo: 0, Offset: 42 is stored on disk
2025-11-27T14:52:23.670283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.670293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.670296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.670299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.670302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.670312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.670326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.670416Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.670771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457483:3485] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 42
    Offset: 42
    WriteTimestampMS: 1764255143666
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.671705Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.672230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457487:3487], now have 1 active actors on pipe
2025-11-27T14:52:23.672325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.672340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.672384Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|75732ddc-b1d48593-79995102-7a654deb_43 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.672404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.672412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.672418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.672420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.672425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.672435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.672439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.672442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.672458Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.672476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.672530Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.672839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457487:3487] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|75732ddc-b1d48593-79995102-7a654deb_43"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|75732ddc-b1d48593-79995102-7a654deb_43"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 43
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.673958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.673966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.673970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.673972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.673974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.675204Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.675647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457490:3488], now have 1 active actors on pipe
2025-11-27T14:52:23.676141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.676156Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.676177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 43 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.676187Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.676203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.676211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.676220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.676223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.676228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.676249Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 43 partNo 0
2025-11-27T14:52:23.676292Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 43 partNo 0 FormedBlobsCount 0 NewHead: Offset 43 PartNo 0 PackedSize 262214 count 1 nextOffset 44 batches 1
2025-11-27T14:52:23.676299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.676302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.676303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.676395Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 43,1 HeadOffset 43 endOffset 43 curOffset 44 d0000000000_00000000000000000043_00000_0000000001_00000? size 262204 WTime 1764255143675
2025-11-27T14:52:23.676441Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.676455Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 43 partNo 0 count 1 size 262204
2025-11-27T14:52:23.678843Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 43 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.678869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.678894Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 43 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.678897Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.678906Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.678913Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 43, partNo: 0, Offset: 43 is stored on disk
2025-11-27T14:52:23.678945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.678953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.678954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.678958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.678960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.678970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.678982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.679074Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.679460Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457490:3488] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 43
    Offset: 43
    WriteTimestampMS: 1764255143675
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.680235Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.680740Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457494:3490], now have 1 active actors on pipe
2025-11-27T14:52:23.680825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.680839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.680873Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|1b40d762-bbc70280-b9d44469-9841343_44 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.680886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.680890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.680894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.680902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.680905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.680915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.680919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.680923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.680937Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.680953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.681009Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.681243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457494:3490] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|1b40d762-bbc70280-b9d44469-9841343_44"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|1b40d762-bbc70280-b9d44469-9841343_44"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 44
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.683678Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.684197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457497:3491], now have 1 active actors on pipe
2025-11-27T14:52:23.684647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.684666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.684696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 44 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.684716Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.684734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.684743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.684753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.684760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.684764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.684797Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 44 partNo 0
2025-11-27T14:52:23.684842Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 44 partNo 0 FormedBlobsCount 0 NewHead: Offset 44 PartNo 0 PackedSize 262214 count 1 nextOffset 45 batches 1
2025-11-27T14:52:23.684854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.684857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.684860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.685004Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 44,1 HeadOffset 44 endOffset 44 curOffset 45 d0000000000_00000000000000000044_00000_0000000001_00000? size 262204 WTime 1764255143684
2025-11-27T14:52:23.685050Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.685069Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 44 partNo 0 count 1 size 262204
2025-11-27T14:52:23.690354Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 44 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.690395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.690422Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.690426Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 44 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.690437Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.690464Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 44, partNo: 0, Offset: 44 is stored on disk
2025-11-27T14:52:23.690510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.690522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.690524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.690527Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.690529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.690539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.690556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.691011Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 44
    Offset: 44
    WriteTimestampMS: 1764255143684
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-27T14:52:23.691462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457497:3491] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.692374Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.692934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457501:3493], now have 1 active actors on pipe
2025-11-27T14:52:23.693062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.693075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.693120Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f4346cc1-6ce9cb87-d65e180d-b5fbb113_45 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.693134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.693138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.693143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.693145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.693149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.693160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.693169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.693174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.693191Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.693206Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.693287Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.693579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457501:3493] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f4346cc1-6ce9cb87-d65e180d-b5fbb113_45"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f4346cc1-6ce9cb87-d65e180d-b5fbb113_45"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 45
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.698231Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.698775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457504:3494], now have 1 active actors on pipe
2025-11-27T14:52:23.699233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.699253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.699282Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 45 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.699307Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.699330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.699340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.699352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.699357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.699363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.699402Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 45 partNo 0
2025-11-27T14:52:23.699474Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 45 partNo 0 FormedBlobsCount 0 NewHead: Offset 45 PartNo 0 PackedSize 262214 count 1 nextOffset 46 batches 1
2025-11-27T14:52:23.699489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.699492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.699496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.699644Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 45,1 HeadOffset 45 endOffset 45 curOffset 46 d0000000000_00000000000000000045_00000_0000000001_00000? size 262204 WTime 1764255143699
2025-11-27T14:52:23.699694Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.699713Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 45 partNo 0 count 1 size 262204
2025-11-27T14:52:23.702403Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 45 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.702440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.702465Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 45 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.702467Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.702480Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.702494Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 45, partNo: 0, Offset: 45 is stored on disk
2025-11-27T14:52:23.702549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.702558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.702560Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.702563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.702565Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.702575Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.702590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.702710Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 45
    Offset: 45
    WriteTimestampMS: 1764255143699
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-27T14:52:23.703185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457504:3494] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.703930Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.704512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457508:3496], now have 1 active actors on pipe
2025-11-27T14:52:23.704647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.704660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.704699Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|8dc5e122-578c35c2-d9a81a44-ce1dcdac_46 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.704715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.704718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.704722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.704724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.704728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.704739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.704742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.704745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.704758Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.704771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.704849Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.705119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457508:3496] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|8dc5e122-578c35c2-d9a81a44-ce1dcdac_46"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|8dc5e122-578c35c2-d9a81a44-ce1dcdac_46"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 46
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.707592Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.708146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457511:3497], now have 1 active actors on pipe
2025-11-27T14:52:23.708578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.708591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.708612Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 46 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.708625Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.708641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.708649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.708660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.708661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.708665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.708694Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 46 partNo 0
2025-11-27T14:52:23.708735Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 46 partNo 0 FormedBlobsCount 0 NewHead: Offset 46 PartNo 0 PackedSize 262214 count 1 nextOffset 47 batches 1
2025-11-27T14:52:23.708745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.708748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.708750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.708865Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 46,1 HeadOffset 46 endOffset 46 curOffset 47 d0000000000_00000000000000000046_00000_0000000001_00000? size 262204 WTime 1764255143708
2025-11-27T14:52:23.708906Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.708919Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 46 partNo 0 count 1 size 262204
2025-11-27T14:52:23.711691Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 46
    Offset: 46
    WriteTimestampMS: 1764255143708
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.711462Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 46 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.711492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.711518Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.711517Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 46 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.711529Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.711538Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 46, partNo: 0, Offset: 46 is stored on disk
2025-11-27T14:52:23.711577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.711580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.711582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.711585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.711587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.711595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.711606Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.712051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457511:3497] destroyed
2025-11-27T14:52:23.712958Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.713368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457515:3499], now have 1 active actors on pipe
2025-11-27T14:52:23.713445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.713455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.713494Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f27cf6d7-f97262bb-d7800254-c76fae10_47 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.713502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.713505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.713597Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.713512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.713514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.713516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.713526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.713528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.713530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.713541Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.713550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.713801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457515:3499] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f27cf6d7-f97262bb-d7800254-c76fae10_47"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f27cf6d7-f97262bb-d7800254-c76fae10_47"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 47
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.716377Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.716955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457518:3500], now have 1 active actors on pipe
2025-11-27T14:52:23.717567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.717594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.717620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 47 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.717635Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.717663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.717670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.717681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.717684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.717690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.717725Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 47 partNo 0
2025-11-27T14:52:23.717777Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 47 partNo 0 FormedBlobsCount 0 NewHead: Offset 47 PartNo 0 PackedSize 262214 count 1 nextOffset 48 batches 1
2025-11-27T14:52:23.717785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.717790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.717794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.717945Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 47,1 HeadOffset 47 endOffset 47 curOffset 48 d0000000000_00000000000000000047_00000_0000000001_00000? size 262204 WTime 1764255143717
2025-11-27T14:52:23.717990Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.718007Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 47 partNo 0 count 1 size 262204
2025-11-27T14:52:23.721718Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 47 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.721747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.721776Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.721793Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.721802Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 47, partNo: 0, Offset: 47 is stored on disk
2025-11-27T14:52:23.721840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.721844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.721846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.721849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.721852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.721861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.721876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.722034Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 47 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.722147Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.722642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457518:3500] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 47
    Offset: 47
    WriteTimestampMS: 1764255143717
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.724093Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.724572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457522:3502], now have 1 active actors on pipe
2025-11-27T14:52:23.724789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.724801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.724838Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|40f6fc94-b597582d-ec9e227d-45798196_48 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.724850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.724854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.724861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.724863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.724868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.724880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.724883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.724887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.724905Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.724923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.725082Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|40f6fc94-b597582d-ec9e227d-45798196_48"
    Status: Active
    SeqNo: 0
  }
}
2025-11-27T14:52:23.725591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457522:3502] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|40f6fc94-b597582d-ec9e227d-45798196_48"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 48
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.727954Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.728423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457525:3503], now have 1 active actors on pipe
2025-11-27T14:52:23.728954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.728966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.728991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 48 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.729001Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.729015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.729018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.729029Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.729032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.729038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.729072Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 48 partNo 0
2025-11-27T14:52:23.729110Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 48 partNo 0 FormedBlobsCount 0 NewHead: Offset 48 PartNo 0 PackedSize 262214 count 1 nextOffset 49 batches 1
2025-11-27T14:52:23.729117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.729120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.729122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.729249Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 48,1 HeadOffset 48 endOffset 48 curOffset 49 d0000000000_00000000000000000048_00000_0000000001_00000? size 262204 WTime 1764255143729
2025-11-27T14:52:23.729287Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.729300Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 48 partNo 0 count 1 size 262204
2025-11-27T14:52:23.733821Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 48 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.733853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.733884Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.733897Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.733914Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 48, partNo: 0, Offset: 48 is stored on disk
2025-11-27T14:52:23.733956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.733960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.733962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.733965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.733967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.733979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.733996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.734063Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 48 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.734162Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 48
    Offset: 48
    WriteTimestampMS: 1764255143728
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-27T14:52:23.734569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457525:3503] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.735356Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.735923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457529:3505], now have 1 active actors on pipe
2025-11-27T14:52:23.736130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.736167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.736225Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|d96c0a9b-c39b8711-fc47f01f-d3b8127_49 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.736244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.736249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.736256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.736259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.736267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.736281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.736284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.736287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.736306Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.736322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.736396Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.736723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457529:3505] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|d96c0a9b-c39b8711-fc47f01f-d3b8127_49"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|d96c0a9b-c39b8711-fc47f01f-d3b8127_49"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 49
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.739257Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.739851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457532:3506], now have 1 active actors on pipe
2025-11-27T14:52:23.740241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.740256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.740278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 49 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.740292Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.740307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.740311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.740320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.740323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.740328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.740354Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 49 partNo 0
2025-11-27T14:52:23.740389Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 49 partNo 0 FormedBlobsCount 0 NewHead: Offset 49 PartNo 0 PackedSize 262214 count 1 nextOffset 50 batches 1
2025-11-27T14:52:23.740398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.740401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.740405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.740529Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 49,1 HeadOffset 49 endOffset 49 curOffset 50 d0000000000_00000000000000000049_00000_0000000001_00000? size 262204 WTime 1764255143740
2025-11-27T14:52:23.740573Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.740584Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 49 partNo 0 count 1 size 262204
2025-11-27T14:52:23.743080Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 49 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.743110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.743127Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 49 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.743133Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.743144Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.743159Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 49, partNo: 0, Offset: 49 is stored on disk
2025-11-27T14:52:23.743197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.743200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.743202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.743204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.743206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.743214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.743227Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.743375Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 49
    Offset: 49
    WriteTimestampMS: 1764255143740
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-27T14:52:23.743824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457532:3506] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.744578Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.745028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457535:3507], now have 1 active actors on pipe
2025-11-27T14:52:23.745197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.745209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.745243Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a290268c-84c659ba-6772b15a-bbd7d18c_50 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.745258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.745262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.745267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.745269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.745273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.745284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.745288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.745292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.745310Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.745325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.745396Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a290268c-84c659ba-6772b15a-bbd7d18c_50"
    Status: Active
    SeqNo: 0
  }
}
2025-11-27T14:52:23.745732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457535:3507] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a290268c-84c659ba-6772b15a-bbd7d18c_50"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 50
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.747985Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.748558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457538:3508], now have 1 active actors on pipe
2025-11-27T14:52:23.749118Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.749137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.749162Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 50 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.749174Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.749192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.749200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.749213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.749215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.749220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.749252Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 50 partNo 0
2025-11-27T14:52:23.749291Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 50 partNo 0 FormedBlobsCount 0 NewHead: Offset 50 PartNo 0 PackedSize 262214 count 1 nextOffset 51 batches 1
2025-11-27T14:52:23.749303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.749306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.749308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.749443Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 50,1 HeadOffset 50 endOffset 50 curOffset 51 d0000000000_00000000000000000050_00000_0000000001_00000? size 262204 WTime 1764255143747
2025-11-27T14:52:23.749483Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.749496Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 50 partNo 0 count 1 size 262204
2025-11-27T14:52:23.752268Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 50 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.752294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.752317Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.752329Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.752340Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 50, partNo: 0, Offset: 50 is stored on disk
2025-11-27T14:52:23.752353Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 50 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.752368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.752370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.752371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.752373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.752374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.752382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.752392Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.752498Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 50
    Offset: 50
    WriteTimestampMS: 1764255143747
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-27T14:52:23.752872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457538:3508] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.753570Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.754005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457542:3510], now have 1 active actors on pipe
2025-11-27T14:52:23.754105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.754116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.754151Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f1d69927-2a6324a-163b126f-5f7db69_51 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.754165Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.754169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.754182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.754186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.754190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.754201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.754208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.754211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.754226Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.754244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.754288Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.754489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457542:3510] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f1d69927-2a6324a-163b126f-5f7db69_51"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f1d69927-2a6324a-163b126f-5f7db69_51"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 51
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.756928Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.759280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457545:3511], now have 1 active actors on pipe
2025-11-27T14:52:23.759429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.759454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.759484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 51 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.759498Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.759513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.759517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.759528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.759529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.759534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.759567Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 51 partNo 0
2025-11-27T14:52:23.759598Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 51 partNo 0 FormedBlobsCount 0 NewHead: Offset 51 PartNo 0 PackedSize 262214 count 1 nextOffset 52 batches 1
2025-11-27T14:52:23.759605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.759608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.759611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.759737Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 51,1 HeadOffset 51 endOffset 51 curOffset 52 d0000000000_00000000000000000051_00000_0000000001_00000? size 262204 WTime 1764255143759
2025-11-27T14:52:23.759771Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.759781Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 51 partNo 0 count 1 size 262204
2025-11-27T14:52:23.762757Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 51 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.762792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.762822Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.762824Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 51 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.762844Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.762856Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 51, partNo: 0, Offset: 51 is stored on disk
2025-11-27T14:52:23.762892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.762901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.762903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.762904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.762906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.762914Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.762927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.763035Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 51
    Offset: 51
    WriteTimestampMS: 1764255143759
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-27T14:52:23.763560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457545:3511] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.764361Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.764858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457549:3513], now have 1 active actors on pipe
2025-11-27T14:52:23.764982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.765002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.765046Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|6e0217ce-b53050b1-14739ba0-6bad8d_52 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.765062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.765072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.765077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.765080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.765084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.765096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.765100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.765103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.765121Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.765142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.765210Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.765462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457549:3513] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|6e0217ce-b53050b1-14739ba0-6bad8d_52"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|6e0217ce-b53050b1-14739ba0-6bad8d_52"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 52
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.767424Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.767923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457552:3514], now have 1 active actors on pipe
2025-11-27T14:52:23.768400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.768419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.768451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 52 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.768469Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.768489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.768497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.768507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.768513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.768518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.768549Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 52 partNo 0
2025-11-27T14:52:23.768595Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 52 partNo 0 FormedBlobsCount 0 NewHead: Offset 52 PartNo 0 PackedSize 262214 count 1 nextOffset 53 batches 1
2025-11-27T14:52:23.768608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.768611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.768617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.768743Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 52,1 HeadOffset 52 endOffset 52 curOffset 53 d0000000000_00000000000000000052_00000_0000000001_00000? size 262204 WTime 1764255143768
2025-11-27T14:52:23.768794Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.768819Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 52 partNo 0 count 1 size 262204
2025-11-27T14:52:23.771702Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 52 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.771736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.771762Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 52 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.771767Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.771779Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.771800Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 52, partNo: 0, Offset: 52 is stored on disk
2025-11-27T14:52:23.771843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.771852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.771854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.771856Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.771858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.771868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.771881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.771970Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.772322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457552:3514] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 52
    Offset: 52
    WriteTimestampMS: 1764255143768
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.772984Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.773474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457556:3516], now have 1 active actors on pipe
2025-11-27T14:52:23.773562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.773572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.773611Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|661e98ef-71678216-53bd205e-90298b1_53 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.773621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.773625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.773633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.773635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.773639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.773651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.773654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.773661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.773677Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.773690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.773751Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.774003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457556:3516] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|661e98ef-71678216-53bd205e-90298b1_53"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
2025-11-27T14:52:23.774321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.774327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.774329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.774331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.774333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|661e98ef-71678216-53bd205e-90298b1_53"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 53
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.776394Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.776926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457559:3517], now have 1 active actors on pipe
2025-11-27T14:52:23.777314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.777325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.777344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 53 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.777354Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.777377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.777381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.777390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.777392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.777397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.777435Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 53 partNo 0
2025-11-27T14:52:23.777471Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 53 partNo 0 FormedBlobsCount 0 NewHead: Offset 53 PartNo 0 PackedSize 262214 count 1 nextOffset 54 batches 1
2025-11-27T14:52:23.777479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.777482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.777485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.777788Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 53,1 HeadOffset 53 endOffset 53 curOffset 54 d0000000000_00000000000000000053_00000_0000000001_00000? size 262204 WTime 1764255143777
2025-11-27T14:52:23.777835Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.777851Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 53 partNo 0 count 1 size 262204
2025-11-27T14:52:23.780227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 53 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.780252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.780274Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.780284Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.780284Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 53 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.780299Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 53, partNo: 0, Offset: 53 is stored on disk
2025-11-27T14:52:23.780330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.780336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.780337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.780339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.780340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.780347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.780358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.780459Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.780869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457559:3517] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 53
    Offset: 53
    WriteTimestampMS: 1764255143777
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.781670Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.782132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457564:3520], now have 1 active actors on pipe
2025-11-27T14:52:23.782236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.782262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.782298Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|28ef596b-dfeda421-9cad7411-c27c48c6_54 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.782312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.782315Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.782320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.782321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.782325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.782336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.782340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.782345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.782358Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.782374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.782444Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.782696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457564:3520] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|28ef596b-dfeda421-9cad7411-c27c48c6_54"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|28ef596b-dfeda421-9cad7411-c27c48c6_54"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 54
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.784853Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.785324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457567:3521], now have 1 active actors on pipe
2025-11-27T14:52:23.785758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.785774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.785801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 54 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.785820Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.785842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.785852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.785864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.785872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.785878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.785910Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 54 partNo 0
2025-11-27T14:52:23.785953Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 54 partNo 0 FormedBlobsCount 0 NewHead: Offset 54 PartNo 0 PackedSize 262214 count 1 nextOffset 55 batches 1
2025-11-27T14:52:23.785965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.785968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.785971Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.786084Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 54,1 HeadOffset 54 endOffset 54 curOffset 55 d0000000000_00000000000000000054_00000_0000000001_00000? size 262204 WTime 1764255143786
2025-11-27T14:52:23.786132Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.786149Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 54 partNo 0 count 1 size 262204
2025-11-27T14:52:23.789093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 54 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.789116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.789129Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 54 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.789140Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.789166Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.789177Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 54, partNo: 0, Offset: 54 is stored on disk
2025-11-27T14:52:23.789202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.789208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.789210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.789211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.789212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.789221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.789233Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.789318Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.789694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457567:3521] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 54
    Offset: 54
    WriteTimestampMS: 1764255143785
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.790557Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.790985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457571:3523], now have 1 active actors on pipe
2025-11-27T14:52:23.791095Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.791104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.791133Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a1cfc11d-c4ab4223-c1234b27-ff8d11a9_55 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.791144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.791147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.791150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.791151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.791154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.791164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.791167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.791168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.791179Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.791187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.791237Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a1cfc11d-c4ab4223-c1234b27-ff8d11a9_55"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a1cfc11d-c4ab4223-c1234b27-ff8d11a9_55"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 55
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.793853Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.795477Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457571:3523] destroyed
2025-11-27T14:52:23.795496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457574:3524], now have 1 active actors on pipe
2025-11-27T14:52:23.795583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.795591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.795609Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 55 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.795624Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.795636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.795643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.795652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.795654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.795658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.795680Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 55 partNo 0
2025-11-27T14:52:23.795712Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 55 partNo 0 FormedBlobsCount 0 NewHead: Offset 55 PartNo 0 PackedSize 262214 count 1 nextOffset 56 batches 1
2025-11-27T14:52:23.795720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.795722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.795724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.795812Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 55,1 HeadOffset 55 endOffset 55 curOffset 56 d0000000000_00000000000000000055_00000_0000000001_00000? size 262204 WTime 1764255143795
2025-11-27T14:52:23.795851Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.795862Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 55 partNo 0 count 1 size 262204
2025-11-27T14:52:23.798228Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 55 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.798257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.798273Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 55 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.798281Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.798293Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.798303Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 55, partNo: 0, Offset: 55 is stored on disk
2025-11-27T14:52:23.798339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.798348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.798350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.798352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.798354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.798363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.798373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.798449Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.798780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457574:3524] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 55
    Offset: 55
    WriteTimestampMS: 1764255143795
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.799499Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.799957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457578:3526], now have 1 active actors on pipe
2025-11-27T14:52:23.800071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.800083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.800121Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|d79c1b99-2fc5f9d1-73f5abfb-b8ef6c5d_56 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.800136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.800139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.800144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.800146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.800151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.800163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.800165Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.800168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.800181Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.800198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.800273Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.800510Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457578:3526] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|d79c1b99-2fc5f9d1-73f5abfb-b8ef6c5d_56"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|d79c1b99-2fc5f9d1-73f5abfb-b8ef6c5d_56"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 56
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.802874Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.803342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457581:3527], now have 1 active actors on pipe
2025-11-27T14:52:23.803826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.803845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.803871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 56 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.803900Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.803921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.803930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.803941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.803948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.803952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.803987Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 56 partNo 0
2025-11-27T14:52:23.804031Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 56 partNo 0 FormedBlobsCount 0 NewHead: Offset 56 PartNo 0 PackedSize 262214 count 1 nextOffset 57 batches 1
2025-11-27T14:52:23.804046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.804051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.804068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.804209Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 56,1 HeadOffset 56 endOffset 56 curOffset 57 d0000000000_00000000000000000056_00000_0000000001_00000? size 262204 WTime 1764255143804
2025-11-27T14:52:23.804256Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.804276Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 56 partNo 0 count 1 size 262204
2025-11-27T14:52:23.806855Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 56 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.806884Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.806904Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 56 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.806910Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.806920Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.806936Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 56, partNo: 0, Offset: 56 is stored on disk
2025-11-27T14:52:23.806976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.806985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.806987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.806990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.806992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.807000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.807013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.807106Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.807470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457581:3527] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 56
    Offset: 56
    WriteTimestampMS: 1764255143803
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.808381Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.808873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457585:3529], now have 1 active actors on pipe
2025-11-27T14:52:23.808975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.808984Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.809019Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|e542156b-3eb75429-87283d1b-a8456f20_57 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.809027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.809031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.809036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.809038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.809042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.809051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.809053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.809059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.809134Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.809070Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.809080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
2025-11-27T14:52:23.809405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457585:3529] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|e542156b-3eb75429-87283d1b-a8456f20_57"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|e542156b-3eb75429-87283d1b-a8456f20_57"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 57
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.811756Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.812231Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457588:3530], now have 1 active actors on pipe
2025-11-27T14:52:23.812644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.812659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.812682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 57 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.812699Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.812711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.812720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.812729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.812731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.812735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.812765Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 57 partNo 0
2025-11-27T14:52:23.812806Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 57 partNo 0 FormedBlobsCount 0 NewHead: Offset 57 PartNo 0 PackedSize 262214 count 1 nextOffset 58 batches 1
2025-11-27T14:52:23.812818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.812821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.812825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.812942Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 57,1 HeadOffset 57 endOffset 57 curOffset 58 d0000000000_00000000000000000057_00000_0000000001_00000? size 262204 WTime 1764255143812
2025-11-27T14:52:23.812994Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.813012Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 57 partNo 0 count 1 size 262204
2025-11-27T14:52:23.815283Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 57 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.815305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.815328Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 57 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.815340Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.815349Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.815363Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 57, partNo: 0, Offset: 57 is stored on disk
2025-11-27T14:52:23.815395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.815403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.815407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.815410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.815412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.815418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.815427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.815510Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.815835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457588:3530] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 57
    Offset: 57
    WriteTimestampMS: 1764255143812
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.816757Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.817161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457593:3533], now have 1 active actors on pipe
2025-11-27T14:52:23.817260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.817274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.817323Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b16771a5-c87b738d-ed793dcb-73de2430_58 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.817338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.817341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.817346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.817350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.817354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.817365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.817367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.817370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.817382Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.817398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.817473Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b16771a5-c87b738d-ed793dcb-73de2430_58"
    Status: Active
    SeqNo: 0
  }
}
2025-11-27T14:52:23.817749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457593:3533] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b16771a5-c87b738d-ed793dcb-73de2430_58"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 58
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.820162Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.820604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457596:3534], now have 1 active actors on pipe
2025-11-27T14:52:23.821038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.821052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.821078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 58 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.821092Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.821107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.821111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.821122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.821123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.821127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.821164Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 58 partNo 0
2025-11-27T14:52:23.821278Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 58 partNo 0 FormedBlobsCount 0 NewHead: Offset 58 PartNo 0 PackedSize 262214 count 1 nextOffset 59 batches 1
2025-11-27T14:52:23.821287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.821289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.821298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.821443Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 58,1 HeadOffset 58 endOffset 58 curOffset 59 d0000000000_00000000000000000058_00000_0000000001_00000? size 262204 WTime 1764255143821
2025-11-27T14:52:23.821488Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.821501Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 58 partNo 0 count 1 size 262204
2025-11-27T14:52:23.824021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 58 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.824075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.824090Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 58 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.824114Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.824128Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.824140Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 58, partNo: 0, Offset: 58 is stored on disk
2025-11-27T14:52:23.824177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.824182Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.824184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.824187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.824190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.824202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.824217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.824306Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.824685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457596:3534] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 58
    Offset: 58
    WriteTimestampMS: 1764255143820
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.825356Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.825712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457600:3536], now have 1 active actors on pipe
2025-11-27T14:52:23.825838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.825854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.825897Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|126ec0f1-4634b1ca-735285c7-a094b73e_59 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.825913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.825921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.825926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.825929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.825934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.825951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.825959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.825963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.825983Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.826002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.826112Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|126ec0f1-4634b1ca-735285c7-a094b73e_59"
    Status: Active
    SeqNo: 0
  }
}
2025-11-27T14:52:23.826397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457600:3536] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|126ec0f1-4634b1ca-735285c7-a094b73e_59"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 59
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.828338Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.828779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457603:3537], now have 1 active actors on pipe
2025-11-27T14:52:23.829172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.829189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.829229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 59 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.829248Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.829269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.829277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.829288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.829295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.829299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.829336Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 59 partNo 0
2025-11-27T14:52:23.829382Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 59 partNo 0 FormedBlobsCount 0 NewHead: Offset 59 PartNo 0 PackedSize 262214 count 1 nextOffset 60 batches 1
2025-11-27T14:52:23.829413Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.829421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.829424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.829558Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 59,1 HeadOffset 59 endOffset 59 curOffset 60 d0000000000_00000000000000000059_00000_0000000001_00000? size 262204 WTime 1764255143829
2025-11-27T14:52:23.829610Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.829628Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 59 partNo 0 count 1 size 262204
2025-11-27T14:52:23.832033Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 59 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.832076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.832102Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 59 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.832104Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.832117Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.832129Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 59, partNo: 0, Offset: 59 is stored on disk
2025-11-27T14:52:23.832167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.832178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.832180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.832183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.832188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.832203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.832221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.832298Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.832639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457603:3537] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 59
    Offset: 59
    WriteTimestampMS: 1764255143829
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.833313Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.833748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457607:3539], now have 1 active actors on pipe
2025-11-27T14:52:23.833840Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.833852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.833889Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|c2d4867e-464c27c9-4ed732e7-793a4d5a_60 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.833903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.833912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.833919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.833921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.833925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.833936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.833943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.833946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.833960Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.833977Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.834029Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.834243Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457607:3539] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|c2d4867e-464c27c9-4ed732e7-793a4d5a_60"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|c2d4867e-464c27c9-4ed732e7-793a4d5a_60"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 60
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.836156Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.836552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457610:3540], now have 1 active actors on pipe
2025-11-27T14:52:23.836995Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.837013Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.837038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 60 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.837056Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.837078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.837087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.837098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.837106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.837110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.837146Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 60 partNo 0
2025-11-27T14:52:23.837194Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 60 partNo 0 FormedBlobsCount 0 NewHead: Offset 60 PartNo 0 PackedSize 262214 count 1 nextOffset 61 batches 1
2025-11-27T14:52:23.837209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.837216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.837220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.837367Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 60,1 HeadOffset 60 endOffset 60 curOffset 61 d0000000000_00000000000000000060_00000_0000000001_00000? size 262204 WTime 1764255143837
2025-11-27T14:52:23.837414Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.837433Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 60 partNo 0 count 1 size 262204
2025-11-27T14:52:23.839748Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 60 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.839798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.839825Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.839834Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 60 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.839837Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.839847Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 60, partNo: 0, Offset: 60 is stored on disk
2025-11-27T14:52:23.839888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.839892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.839894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.839897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.839900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.839911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.839924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.840031Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.840370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457610:3540] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 60
    Offset: 60
    WriteTimestampMS: 1764255143836
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.841070Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.841511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457614:3542], now have 1 active actors on pipe
2025-11-27T14:52:23.841600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.841616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.841656Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|3a7c9434-ff33ec36-c31ece35-15e1db3e_61 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.841672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.841677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.841683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.841690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.841694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.841705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.841713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.841716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.841732Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.841750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.841808Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.842047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457614:3542] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|3a7c9434-ff33ec36-c31ece35-15e1db3e_61"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|3a7c9434-ff33ec36-c31ece35-15e1db3e_61"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 61
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.844390Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.844815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457617:3543], now have 1 active actors on pipe
2025-11-27T14:52:23.845235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.845256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.845288Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 61 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.845307Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.845330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.845339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.845352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.845359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.845364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.845401Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 61 partNo 0
2025-11-27T14:52:23.845446Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 61 partNo 0 FormedBlobsCount 0 NewHead: Offset 61 PartNo 0 PackedSize 262214 count 1 nextOffset 62 batches 1
2025-11-27T14:52:23.845460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.845463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.845467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.845610Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 61,1 HeadOffset 61 endOffset 61 curOffset 62 d0000000000_00000000000000000061_00000_0000000001_00000? size 262204 WTime 1764255143845
2025-11-27T14:52:23.845658Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.845678Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 61 partNo 0 count 1 size 262204
2025-11-27T14:52:23.848108Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 61 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.848142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.848165Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 61 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.848173Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.848186Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.848205Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 61, partNo: 0, Offset: 61 is stored on disk
2025-11-27T14:52:23.848245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.848259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.848262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.848269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.848271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.848285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.848318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.848398Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 61
    Offset: 61
    WriteTimestampMS: 1764255143844
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-27T14:52:23.848818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457617:3543] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.850677Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.851286Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457621:3545], now have 1 active actors on pipe
2025-11-27T14:52:23.851429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.851459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.851500Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|118a9eca-710489ab-b72bd686-aca8fdd8_62 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.851520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.851527Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.851532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.851535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.851539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.851550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.851553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.851557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.851577Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.851600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.851682Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.852037Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457621:3545] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|118a9eca-710489ab-b72bd686-aca8fdd8_62"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|118a9eca-710489ab-b72bd686-aca8fdd8_62"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 62
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.857068Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.857611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457626:3547], now have 1 active actors on pipe
2025-11-27T14:52:23.858067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.858087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.858117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 62 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.858142Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.858161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.858172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.858187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.858196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.858202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.858240Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 62 partNo 0
2025-11-27T14:52:23.858289Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 62 partNo 0 FormedBlobsCount 0 NewHead: Offset 62 PartNo 0 PackedSize 262214 count 1 nextOffset 63 batches 1
2025-11-27T14:52:23.858304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.858308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.858312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.858448Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 62,1 HeadOffset 62 endOffset 62 curOffset 63 d0000000000_00000000000000000062_00000_0000000001_00000? size 262204 WTime 1764255143858
2025-11-27T14:52:23.858505Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.858525Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 62 partNo 0 count 1 size 262204
2025-11-27T14:52:23.861365Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 62 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.861404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.861438Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.861458Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.861470Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 62, partNo: 0, Offset: 62 is stored on disk
2025-11-27T14:52:23.861523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.861533Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.861547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.861549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.861552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.861558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.861570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.861586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.861715Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.862053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457626:3547] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 62
    Offset: 62
    WriteTimestampMS: 1764255143857
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.863139Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.863655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457630:3549], now have 1 active actors on pipe
2025-11-27T14:52:23.863737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.863752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.863796Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|58308cdd-f62a8076-a3f4bfe3-46000f8d_63 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.863812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.863816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.863823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.863826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.863830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.863842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.863850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.863853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.863871Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.863892Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.863953Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|58308cdd-f62a8076-a3f4bfe3-46000f8d_63"
    Status: Active
    SeqNo: 0
  }
}
2025-11-27T14:52:23.864193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457630:3549] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|58308cdd-f62a8076-a3f4bfe3-46000f8d_63"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 63
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.866388Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.866743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457633:3550], now have 1 active actors on pipe
2025-11-27T14:52:23.867190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.867201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.867219Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 63 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.867231Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.867241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.867244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.867252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.867259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.867263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.867283Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 63 partNo 0
2025-11-27T14:52:23.867316Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 63 partNo 0 FormedBlobsCount 0 NewHead: Offset 63 PartNo 0 PackedSize 262214 count 1 nextOffset 64 batches 1
2025-11-27T14:52:23.867326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.867329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.867331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.867477Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 63,1 HeadOffset 63 endOffset 63 curOffset 64 d0000000000_00000000000000000063_00000_0000000001_00000? size 262204 WTime 1764255143867
2025-11-27T14:52:23.867520Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.867534Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 63 partNo 0 count 1 size 262204
2025-11-27T14:52:23.869789Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 63 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.869813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.869831Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.869839Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.869840Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 63 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.869847Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 63, partNo: 0, Offset: 63 is stored on disk
2025-11-27T14:52:23.869872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.869890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.869892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.869895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.869896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.869905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000032_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000033_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000034_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000035_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869918Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000036_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869921Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000037_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000038_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000039_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000040_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000041_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000042_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000043_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000044_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000045_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000046_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000047_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000048_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869954Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000049_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000050_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000051_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869962Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000052_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000053_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000054_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000055_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000056_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000057_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000058_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869981Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000059_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000060_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000061_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000062_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000063_00000_0000000001_00000? (262204)
2025-11-27T14:52:23.869993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 32 keys were taken away. Let's read 8390528 bytes
2025-11-27T14:52:23.870002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.870018Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 32 blobs
2025-11-27T14:52:23.870026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000032_00000_0000000001_00000?
2025-11-27T14:52:23.870037Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000033_00000_0000000001_00000?
2025-11-27T14:52:23.870049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000034_00000_0000000001_00000?
2025-11-27T14:52:23.870061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000035_00000_0000000001_00000?
2025-11-27T14:52:23.870074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000036_00000_0000000001_00000?
2025-11-27T14:52:23.870097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000037_00000_0000000001_00000?
2025-11-27T14:52:23.870110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000038_00000_0000000001_00000?
2025-11-27T14:52:23.870119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000039_00000_0000000001_00000?
2025-11-27T14:52:23.870128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000040_00000_0000000001_00000?
2025-11-27T14:52:23.870112Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.870159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000041_00000_0000000001_00000?
2025-11-27T14:52:23.870173Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000042_00000_0000000001_00000?
2025-11-27T14:52:23.870181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000043_00000_0000000001_00000?
2025-11-27T14:52:23.870189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000044_00000_0000000001_00000?
2025-11-27T14:52:23.870197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000045_00000_0000000001_00000?
2025-11-27T14:52:23.870205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000046_00000_0000000001_00000?
2025-11-27T14:52:23.870213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000047_00000_0000000001_00000?
2025-11-27T14:52:23.870222Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000048_00000_0000000001_00000?
2025-11-27T14:52:23.870266Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000049_00000_0000000001_00000?
2025-11-27T14:52:23.870278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000050_00000_0000000001_00000?
2025-11-27T14:52:23.870287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000051_00000_0000000001_00000?
2025-11-27T14:52:23.870296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000052_00000_0000000001_00000?
2025-11-27T14:52:23.870309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000053_00000_0000000001_00000?
2025-11-27T14:52:23.870317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000054_00000_0000000001_00000?
2025-11-27T14:52:23.870326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000055_00000_0000000001_00000?
2025-11-27T14:52:23.870340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000056_00000_0000000001_00000?
2025-11-27T14:52:23.870347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000057_00000_0000000001_00000?
2025-11-27T14:52:23.870357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000058_00000_0000000001_00000?
2025-11-27T14:52:23.870380Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000059_00000_0000000001_00000?
2025-11-27T14:52:23.870393Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000060_00000_0000000001_00000?
2025-11-27T14:52:23.870405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000061_00000_0000000001_00000?
2025-11-27T14:52:23.870418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000062_00000_0000000001_00000?
2025-11-27T14:52:23.870431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000063_00000_0000000001_00000?
2025-11-27T14:52:23.870446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 32 blobs for compaction
2025-11-27T14:52:23.870454Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 32 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870458Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 33 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870461Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 34 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 35 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870467Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 36 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870471Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 37 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870474Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 38 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870478Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 39 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870480Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 40 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870483Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 41 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870487Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 42 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870489Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 43 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870493Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 44 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870496Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 45 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870499Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 46 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870503Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 47 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870508Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 48 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870511Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 49 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870515Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 50 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870518Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 51 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 52 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 53 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870537Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 54 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 55 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870544Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 56 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870547Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 57 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870549Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 58 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870552Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 59 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870555Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 60 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870558Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 61 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870561Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 62 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 63 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:23.870581Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 32 blobs are from cache.
2025-11-27T14:52:23.870603Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 32 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870606Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 33 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870609Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 34 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870612Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 35 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870616Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 36 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457633:3550] destroyed
2025-11-27T14:52:23.870620Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 37 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-27T14:52:23.870624Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 38 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/32] d0000000000_00000000000000000032_00000_0000000001_00000?
2025-11-27T14:52:23.870629Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.870629Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 39 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000032_00000_0000000001_00000?, parameters.CurOffset=32
2025-11-27T14:52:23.870633Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 40 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870636Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 41 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870640Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 42 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870643Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 43 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870648Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 44 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870651Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 45 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870654Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 46 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870658Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 47 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870662Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 48 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870666Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 49 partno 0 count 1 parts 0 suffix '63'
CallPersQueueGRPC response:
2025-11-27T14:52:23.870670Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 50 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870673Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 51 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870677Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 52 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870681Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 53 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870695Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 54 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870699Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 55 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870702Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 56 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870705Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 57 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870708Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 58 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870711Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 59 partno 0 count 1 parts 0 suffix '63'
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 63
    Offset: 63
    WriteTimestampMS: 1764255143866
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-27T14:52:23.870715Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 60 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870718Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 61 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870721Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870723Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 63 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:23.870741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 32.0/1
2025-11-27T14:52:23.870748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 32 partNo 0
2025-11-27T14:52:23.870766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 32 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 262214 count 1 nextOffset 33 batches 1
2025-11-27T14:52:23.870769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 32:0 appended
2025-11-27T14:52:23.870774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/32] d0000000000_00000000000000000033_00000_0000000001_00000?
2025-11-27T14:52:23.870775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.870777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000033_00000_0000000001_00000?, parameters.CurOffset=33
2025-11-27T14:52:23.870894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 33.0/1
2025-11-27T14:52:23.870898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 33 partNo 0
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.871103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 33 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 524387 count 2 nextOffset 34 batches 1
2025-11-27T14:52:23.871117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 33:0 appended
2025-11-27T14:52:23.871121Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/32] d0000000000_00000000000000000034_00000_0000000001_00000?
2025-11-27T14:52:23.871123Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.871125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000034_00000_0000000001_00000?, parameters.CurOffset=34
2025-11-27T14:52:23.871178Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 34.0/1
2025-11-27T14:52:23.871188Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 34 partNo 0
2025-11-27T14:52:23.871199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 34 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 786601 count 3 nextOffset 35 batches 2
2025-11-27T14:52:23.871201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 34:0 appended
2025-11-27T14:52:23.871205Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/32] d0000000000_00000000000000000035_00000_0000000001_00000?
2025-11-27T14:52:23.871208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.871211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000035_00000_0000000001_00000?, parameters.CurOffset=35
2025-11-27T14:52:23.871272Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 35.0/1
2025-11-27T14:52:23.871282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 35 partNo 0
2025-11-27T14:52:23.871482Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 35 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 1048774 count 4 nextOffset 36 batches 2
2025-11-27T14:52:23.871494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 35:0 appended
2025-11-27T14:52:23.871498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/32] d0000000000_00000000000000000036_00000_0000000001_00000?
2025-11-27T14:52:23.871501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.871503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000036_00000_0000000001_00000?, parameters.CurOffset=36
2025-11-27T14:52:23.871561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 36.0/1
2025-11-27T14:52:23.871571Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 36 partNo 0
2025-11-27T14:52:23.871582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 36 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 1310988 count 5 nextOffset 37 batches 3
2025-11-27T14:52:23.871584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 36:0 appended
2025-11-27T14:52:23.871588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/32] d0000000000_00000000000000000037_00000_0000000001_00000?
2025-11-27T14:52:23.871590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.871592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000037_00000_0000000001_00000?, parameters.CurOffset=37
2025-11-27T14:52:23.871641Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 37.0/1
2025-11-27T14:52:23.871650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 37 partNo 0
2025-11-27T14:52:23.871637Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.871752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 37 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 1573161 count 6 nextOffset 38 batches 3
2025-11-27T14:52:23.871763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 37:0 appended
2025-11-27T14:52:23.871767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/32] d0000000000_00000000000000000038_00000_0000000001_00000?
2025-11-27T14:52:23.871769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.871773Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000038_00000_0000000001_00000?, parameters.CurOffset=38
2025-11-27T14:52:23.871837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 38.0/1
2025-11-27T14:52:23.871846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 38 partNo 0
2025-11-27T14:52:23.871857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 38 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 1835375 count 7 nextOffset 39 batches 4
2025-11-27T14:52:23.871859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 38:0 appended
2025-11-27T14:52:23.871864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/32] d0000000000_00000000000000000039_00000_0000000001_00000?
2025-11-27T14:52:23.871865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.871868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000039_00000_0000000001_00000?, parameters.CurOffset=39
2025-11-27T14:52:23.871923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 39.0/1
2025-11-27T14:52:23.871932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 39 partNo 0
2025-11-27T14:52:23.872029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 39 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 2097548 count 8 nextOffset 40 batches 4
2025-11-27T14:52:23.872041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 39:0 appended
2025-11-27T14:52:23.872045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[8/32] d0000000000_00000000000000000040_00000_0000000001_00000?
2025-11-27T14:52:23.872047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.872050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000040_00000_0000000001_00000?, parameters.CurOffset=40
2025-11-27T14:52:23.872120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 40.0/1
2025-11-27T14:52:23.872130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 40 partNo 0
2025-11-27T14:52:23.872140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 40 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 2359762 count 9 nextOffset 41 batches 5
2025-11-27T14:52:23.872142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 40:0 appended
2025-11-27T14:52:23.872146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[9/32] d0000000000_00000000000000000041_00000_0000000001_00000?
2025-11-27T14:52:23.872148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.872150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000041_00000_0000000001_00000?, parameters.CurOffset=41
2025-11-27T14:52:23.872209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 41.0/1
2025-11-27T14:52:23.872218Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 41 partNo 0
2025-11-27T14:52:23.872317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 41 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 2621935 count 10 nextOffset 42 batches 5
2025-11-27T14:52:23.872327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 41:0 appended
2025-11-27T14:52:23.872332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[10/32] d0000000000_00000000000000000042_00000_0000000001_00000?
2025-11-27T14:52:23.872333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.872335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000042_00000_0000000001_00000?, parameters.CurOffset=42
2025-11-27T14:52:23.872402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 42.0/1
2025-11-27T14:52:23.872413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 42 partNo 0
2025-11-27T14:52:23.872447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 42 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 2884149 count 11 nextOffset 43 batches 6
2025-11-27T14:52:23.872449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 42:0 appended
2025-11-27T14:52:23.872453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[11/32] d0000000000_00000000000000000043_00000_0000000001_00000?
2025-11-27T14:52:23.872455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.872458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000043_00000_0000000001_00000?, parameters.CurOffset=43
2025-11-27T14:52:23.872507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 43.0/1
2025-11-27T14:52:23.872511Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 43 partNo 0
2025-11-27T14:52:23.872606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 43 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 3146322 count 12 nextOffset 44 batches 6
2025-11-27T14:52:23.872608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 43:0 appended
2025-11-27T14:52:23.872613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[12/32] d0000000000_00000000000000000044_00000_0000000001_00000?
2025-11-27T14:52:23.872614Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.872617Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000044_00000_0000000001_00000?, parameters.CurOffset=44
2025-11-27T14:52:23.872667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 44.0/1
2025-11-27T14:52:23.872670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 44 partNo 0
2025-11-27T14:52:23.872679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 44 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 3408536 count 13 nextOffset 45 batches 7
2025-11-27T14:52:23.872681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 44:0 appended
2025-11-27T14:52:23.872685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[13/32] d0000000000_00000000000000000045_00000_0000000001_00000?
2025-11-27T14:52:23.872687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.872689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000045_00000_0000000001_00000?, parameters.CurOffset=45
2025-11-27T14:52:23.872755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 45.0/1
2025-11-27T14:52:23.872758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 45 partNo 0
2025-11-27T14:52:23.872850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 45 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 3670709 count 14 nextOffset 46 batches 7
2025-11-27T14:52:23.872852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 45:0 appended
2025-11-27T14:52:23.872856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[14/32] d0000000000_00000000000000000046_00000_0000000001_00000?
2025-11-27T14:52:23.872857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.872860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000046_00000_0000000001_00000?, parameters.CurOffset=46
2025-11-27T14:52:23.872912Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 46.0/1
2025-11-27T14:52:23.872915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 46 partNo 0
2025-11-27T14:52:23.872922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 46 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 3932923 count 15 nextOffset 47 batches 8
2025-11-27T14:52:23.872924Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 46:0 appended
2025-11-27T14:52:23.872926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[15/32] d0000000000_00000000000000000047_00000_0000000001_00000?
2025-11-27T14:52:23.872928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.872931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000047_00000_0000000001_00000?, parameters.CurOffset=47
2025-11-27T14:52:23.872980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 47.0/1
2025-11-27T14:52:23.872983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 47 partNo 0
2025-11-27T14:52:23.873074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 47 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 4195096 count 16 nextOffset 48 batches 8
2025-11-27T14:52:23.873077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 47:0 appended
2025-11-27T14:52:23.873085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[16/32] d0000000000_00000000000000000048_00000_0000000001_00000?
2025-11-27T14:52:23.873087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.873090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000048_00000_0000000001_00000?, parameters.CurOffset=48
2025-11-27T14:52:23.873152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 48.0/1
2025-11-27T14:52:23.873155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 48 partNo 0
2025-11-27T14:52:23.873163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 48 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 4457310 count 17 nextOffset 49 batches 9
2025-11-27T14:52:23.873165Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 48:0 appended
2025-11-27T14:52:23.873168Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[17/32] d0000000000_00000000000000000049_00000_0000000001_00000?
2025-11-27T14:52:23.873170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.873175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000049_00000_0000000001_00000?, parameters.CurOffset=49
2025-11-27T14:52:23.873232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 49.0/1
2025-11-27T14:52:23.873237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 49 partNo 0
2025-11-27T14:52:23.873333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 49 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 4719483 count 18 nextOffset 50 batches 9
2025-11-27T14:52:23.873335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 49:0 appended
2025-11-27T14:52:23.873338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[18/32] d0000000000_00000000000000000050_00000_0000000001_00000?
2025-11-27T14:52:23.873340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.873342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000050_00000_0000000001_00000?, parameters.CurOffset=50
2025-11-27T14:52:23.873405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 50.0/1
2025-11-27T14:52:23.873409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 50 partNo 0
2025-11-27T14:52:23.873416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 50 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 4981697 count 19 nextOffset 51 batches 10
2025-11-27T14:52:23.873418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 50:0 appended
2025-11-27T14:52:23.873420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[19/32] d0000000000_00000000000000000051_00000_0000000001_00000?
2025-11-27T14:52:23.873424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.873426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000051_00000_0000000001_00000?, parameters.CurOffset=51
2025-11-27T14:52:23.873484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 51.0/1
2025-11-27T14:52:23.873486Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 51 partNo 0
2025-11-27T14:52:23.873585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 51 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 5243870 count 20 nextOffset 52 batches 10
2025-11-27T14:52:23.873587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 51:0 appended
2025-11-27T14:52:23.873591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[20/32] d0000000000_00000000000000000052_00000_0000000001_00000?
2025-11-27T14:52:23.873593Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.873595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000052_00000_0000000001_00000?, parameters.CurOffset=52
2025-11-27T14:52:23.873662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 52.0/1
2025-11-27T14:52:23.873665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 52 partNo 0
2025-11-27T14:52:23.873674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 52 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 5506084 count 21 nextOffset 53 batches 11
2025-11-27T14:52:23.873676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 52:0 appended
2025-11-27T14:52:23.873679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[21/32] d0000000000_00000000000000000053_00000_0000000001_00000?
2025-11-27T14:52:23.873681Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.873684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000053_00000_0000000001_00000?, parameters.CurOffset=53
2025-11-27T14:52:23.873748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 53.0/1
2025-11-27T14:52:23.873752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 53 partNo 0
2025-11-27T14:52:23.873845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 53 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 5768257 count 22 nextOffset 54 batches 11
2025-11-27T14:52:23.873848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 53:0 appended
2025-11-27T14:52:23.873852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[22/32] d0000000000_00000000000000000054_00000_0000000001_00000?
2025-11-27T14:52:23.873856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.873860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000054_00000_0000000001_00000?, parameters.CurOffset=54
2025-11-27T14:52:23.873927Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 54.0/1
2025-11-27T14:52:23.873931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 54 partNo 0
2025-11-27T14:52:23.873940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 54 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 6030471 count 23 nextOffset 55 batches 12
2025-11-27T14:52:23.873942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 54:0 appended
2025-11-27T14:52:23.873945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[23/32] d0000000000_00000000000000000055_00000_0000000001_00000?
2025-11-27T14:52:23.873947Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.873950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000055_00000_0000000001_00000?, parameters.CurOffset=55
2025-11-27T14:52:23.874011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 55.0/1
2025-11-27T14:52:23.874014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 55 partNo 0
2025-11-27T14:52:23.874108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 55 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 6292644 count 24 nextOffset 56 batches 12
2025-11-27T14:52:23.874110Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 55:0 appended
2025-11-27T14:52:23.874115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[24/32] d0000000000_00000000000000000056_00000_0000000001_00000?
2025-11-27T14:52:23.874117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.874119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000056_00000_0000000001_00000?, parameters.CurOffset=56
2025-11-27T14:52:23.874187Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 56.0/1
2025-11-27T14:52:23.874204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 56 partNo 0
2025-11-27T14:52:23.874214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 56 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 6554858 count 25 nextOffset 57 batches 13
2025-11-27T14:52:23.874216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 56:0 appended
2025-11-27T14:52:23.874220Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[25/32] d0000000000_00000000000000000057_00000_0000000001_00000?
2025-11-27T14:52:23.874222Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.874224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000057_00000_0000000001_00000?, parameters.CurOffset=57
2025-11-27T14:52:23.874282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 57.0/1
2025-11-27T14:52:23.874293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 57 partNo 0
2025-11-27T14:52:23.874775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 57 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 6817031 count 26 nextOffset 58 batches 13
2025-11-27T14:52:23.874788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 57:0 appended
2025-11-27T14:52:23.874793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[26/32] d0000000000_00000000000000000058_00000_0000000001_00000?
2025-11-27T14:52:23.874795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.874797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000058_00000_0000000001_00000?, parameters.CurOffset=58
2025-11-27T14:52:23.874863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 58.0/1
2025-11-27T14:52:23.874875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 58 partNo 0
2025-11-27T14:52:23.874893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 58 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 7079245 count 27 nextOffset 59 batches 14
2025-11-27T14:52:23.874902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 58:0 appended
2025-11-27T14:52:23.874905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[27/32] d0000000000_00000000000000000059_00000_0000000001_00000?
2025-11-27T14:52:23.874908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.874910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000059_00000_0000000001_00000?, parameters.CurOffset=59
2025-11-27T14:52:23.874966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 59.0/1
2025-11-27T14:52:23.874976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 59 partNo 0
2025-11-27T14:52:23.875480Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 59 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 7341418 count 28 nextOffset 60 batches 14
2025-11-27T14:52:23.875492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 59:0 appended
2025-11-27T14:52:23.875497Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[28/32] d0000000000_00000000000000000060_00000_0000000001_00000?
2025-11-27T14:52:23.875499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.875503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000060_00000_0000000001_00000?, parameters.CurOffset=60
2025-11-27T14:52:23.875570Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 60.0/1
2025-11-27T14:52:23.875574Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 60 partNo 0
2025-11-27T14:52:23.875589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 60 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 7603632 count 29 nextOffset 61 batches 15
2025-11-27T14:52:23.875591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 60:0 appended
2025-11-27T14:52:23.875594Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[29/32] d0000000000_00000000000000000061_00000_0000000001_00000?
2025-11-27T14:52:23.875596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.875598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000061_00000_0000000001_00000?, parameters.CurOffset=61
2025-11-27T14:52:23.875656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 61.0/1
2025-11-27T14:52:23.875659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 61 partNo 0
2025-11-27T14:52:23.876158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 61 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 7865805 count 30 nextOffset 62 batches 15
2025-11-27T14:52:23.876170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 61:0 appended
2025-11-27T14:52:23.876175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[30/32] d0000000000_00000000000000000062_00000_0000000001_00000?
2025-11-27T14:52:23.876177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.876180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000062_00000_0000000001_00000?, parameters.CurOffset=62
2025-11-27T14:52:23.876235Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 62.0/1
2025-11-27T14:52:23.876247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 62 partNo 0
2025-11-27T14:52:23.883127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 62 partNo 0 result is x0000000000_00000000000000000031_00000_0000000031_00000 size 8128009
2025-11-27T14:52:23.883153Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1104: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000031_00000_0000000031_00000 new key d0000000000_00000000000000000031_00000_0000000031_00000 size 8128009 WTime 1764255143882
2025-11-27T14:52:23.883185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 62 partNo 0 FormedBlobsCount 1 NewHead: Offset 62 PartNo 0 PackedSize 262214 count 1 nextOffset 63 batches 1
2025-11-27T14:52:23.883192Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 62:0 appended
2025-11-27T14:52:23.883202Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[31/32] d0000000000_00000000000000000063_00000_0000000001_00000?
2025-11-27T14:52:23.883206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:23.883210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000063_00000_0000000001_00000?, parameters.CurOffset=63
2025-11-27T14:52:23.883301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 63.0/1
2025-11-27T14:52:23.883320Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 63 partNo 0
2025-11-27T14:52:23.883879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 63 partNo 0 FormedBlobsCount 0 NewHead: Offset 62 PartNo 0 PackedSize 524387 count 2 nextOffset 64 batches 1
2025-11-27T14:52:23.883893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 63:0 appended
2025-11-27T14:52:23.883941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 62,2 HeadOffset 31 endOffset 32 curOffset 64 d0000000000_00000000000000000062_00000_0000000002_00000| size 524387 WTime 1764255143883
2025-11-27T14:52:23.884301Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457637:3552], now have 1 active actors on pipe
2025-11-27T14:52:23.884324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.884330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.884351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.884360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.884362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.884365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.884367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.884378Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.884390Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 31 partNo 0 count 31 size 8128009
2025-11-27T14:52:23.884402Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 62 partNo 0 count 2 size 524387
2025-11-27T14:52:23.884432Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|c7966c88-9eb86da-6a4ecf3e-ec2e7ae_64 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.884442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.884445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.884448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.884452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.884456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.884463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.884465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.884469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.884477Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.884520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.884725Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.885148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457637:3552] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|c7966c88-9eb86da-6a4ecf3e-ec2e7ae_64"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|c7966c88-9eb86da-6a4ecf3e-ec2e7ae_64"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 64
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.888105Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.888662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457643:3555], now have 1 active actors on pipe
2025-11-27T14:52:23.889129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.889146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.889166Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 64 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.889178Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.889191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.889194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.889203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.889205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.889209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.889230Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 64 partNo 0
2025-11-27T14:52:23.889263Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 64 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 262214 count 1 nextOffset 65 batches 1
2025-11-27T14:52:23.889273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.889275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.889286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.889542Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 64,1 HeadOffset 64 endOffset 64 curOffset 65 d0000000000_00000000000000000064_00000_0000000001_00000? size 262204 WTime 1764255143889
2025-11-27T14:52:23.889587Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.889597Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 64 partNo 0 count 1 size 262204
2025-11-27T14:52:23.897671Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 31 count 31 size 8128009 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.897688Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 62 count 2 size 524387 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.897706Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 64 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.897715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.897725Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 31 parts 0 suffix '0' size 8128009
2025-11-27T14:52:23.897732Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 2 parts 0 suffix '124' size 524387
2025-11-27T14:52:23.897740Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 64 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.898048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-27T14:52:23.898072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.898079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.898103Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.898114Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.898124Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 64, partNo: 0, Offset: 64 is stored on disk
2025-11-27T14:52:23.898152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.898163Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.898167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.898169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.898171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.898228Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.898250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.898273Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.898288Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000032_00000_0000000001_00000?(+) to d0000000000_00000000000000000032_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898296Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000033_00000_0000000001_00000?(+) to d0000000000_00000000000000000033_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898299Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000034_00000_0000000001_00000?(+) to d0000000000_00000000000000000034_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898301Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000035_00000_0000000001_00000?(+) to d0000000000_00000000000000000035_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898304Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000036_00000_0000000001_00000?(+) to d0000000000_00000000000000000036_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898309Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000037_00000_0000000001_00000?(+) to d0000000000_00000000000000000037_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898312Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000038_00000_0000000001_00000?(+) to d0000000000_00000000000000000038_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898315Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000039_00000_0000000001_00000?(+) to d0000000000_00000000000000000039_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898317Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000040_00000_0000000001_00000?(+) to d0000000000_00000000000000000040_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898320Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000041_00000_0000000001_00000?(+) to d0000000000_00000000000000000041_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898322Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000042_00000_0000000001_00000?(+) to d0000000000_00000000000000000042_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898324Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000043_00000_0000000001_00000?(+) to d0000000000_00000000000000000043_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898327Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000044_00000_0000000001_00000?(+) to d0000000000_00000000000000000044_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898330Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000045_00000_0000000001_00000?(+) to d0000000000_00000000000000000045_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898332Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000046_00000_0000000001_00000?(+) to d0000000000_00000000000000000046_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898336Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000047_00000_0000000001_00000?(+) to d0000000000_00000000000000000047_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898339Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000048_00000_0000000001_00000?(+) to d0000000000_00000000000000000048_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898341Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000049_00000_0000000001_00000?(+) to d0000000000_00000000000000000049_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898345Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000050_00000_0000000001_00000?(+) to d0000000000_00000000000000000050_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898348Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000051_00000_0000000001_00000?(+) to d0000000000_00000000000000000051_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898351Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000052_00000_0000000001_00000?(+) to d0000000000_00000000000000000052_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898353Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000053_00000_0000000001_00000?(+) to d0000000000_00000000000000000053_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898355Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000054_00000_0000000001_00000?(+) to d0000000000_00000000000000000054_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898358Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000055_00000_0000000001_00000?(+) to d0000000000_00000000000000000055_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898361Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000056_00000_0000000001_00000?(+) to d0000000000_00000000000000000056_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898363Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000057_00000_0000000001_00000?(+) to d0000000000_00000000000000000057_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898366Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000058_00000_0000000001_00000?(+) to d0000000000_00000000000000000058_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898368Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000059_00000_0000000001_00000?(+) to d0000000000_00000000000000000059_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898371Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000060_00000_0000000001_00000?(+) to d0000000000_00000000000000000060_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898373Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000061_00000_0000000001_00000?(+) to d0000000000_00000000000000000061_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898376Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000062_00000_0000000001_00000?(+) to d0000000000_00000000000000000062_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898378Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000063_00000_0000000001_00000?(+) to d0000000000_00000000000000000063_00000_0000000001_00000?(+)
2025-11-27T14:52:23.898381Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000031_00000_0000000001_00000|(+) to d0000000000_00000000000000000031_00000_0000000001_00000|(+)
2025-11-27T14:52:23.898370Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.898700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457643:3555] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 64
    Offset: 64
    WriteTimestampMS: 1764255143888
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 9
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.899404Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.899472Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 32 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899487Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 33 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899492Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 34 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899496Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 35 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899502Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 36 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899507Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 37 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899512Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 38 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899518Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 39 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899524Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 40 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899527Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 41 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899532Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 42 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899536Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 43 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899541Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 44 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899546Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 45 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899551Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 46 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 47 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899562Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 48 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899566Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 49 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899570Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 50 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899575Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 51 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899579Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 52 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899584Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 53 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899589Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 54 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899603Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 55 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899607Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 56 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899611Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 57 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899614Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 58 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899619Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 59 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899622Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 60 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899626Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 61 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899630Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 62 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899634Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 63 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899640Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 31 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.899663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.899668Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 32 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899676Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 33 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899680Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 34 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899682Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.899684Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 35 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899688Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 36 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.899692Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 37 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.899695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.899696Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 38 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.899699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.899701Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 39 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899705Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 40 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899707Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.899709Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 41 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899713Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 42 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899720Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 43 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899733Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 44 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899737Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 45 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899743Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 46 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899748Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 47 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899752Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 48 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899756Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 49 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899760Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 50 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899763Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 51 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899767Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 52 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899770Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 53 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899774Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 54 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899778Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 55 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899782Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 56 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899785Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 57 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899789Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 58 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899793Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 59 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899796Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 60 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899800Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 61 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899803Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899807Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 63 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.899810Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 0 suffix '124' size 262204
2025-11-27T14:52:23.899889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457649:3559], now have 1 active actors on pipe
2025-11-27T14:52:23.900001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.900015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.900096Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|888a5c9-8b4ef34a-9f5d249b-a4dbf4b6_65 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.900108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.900111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.900116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.900118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.900122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.900130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.900139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.900142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.900155Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.900173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.900276Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.900567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457649:3559] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|888a5c9-8b4ef34a-9f5d249b-a4dbf4b6_65"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|888a5c9-8b4ef34a-9f5d249b-a4dbf4b6_65"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 65
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.902598Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.903071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457653:3561], now have 1 active actors on pipe
2025-11-27T14:52:23.903637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.903661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.903688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 65 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.903707Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.903728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.903739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.903751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.903761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.903765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.903795Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 65 partNo 0
2025-11-27T14:52:23.903852Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 65 partNo 0 FormedBlobsCount 0 NewHead: Offset 65 PartNo 0 PackedSize 262214 count 1 nextOffset 66 batches 1
2025-11-27T14:52:23.903867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.903871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.903874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.904203Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 65,1 HeadOffset 65 endOffset 65 curOffset 66 d0000000000_00000000000000000065_00000_0000000001_00000? size 262204 WTime 1764255143903
2025-11-27T14:52:23.904260Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.904273Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 65 partNo 0 count 1 size 262204
2025-11-27T14:52:23.906763Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 65 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.906800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.906822Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.906832Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.906846Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 65, partNo: 0, Offset: 65 is stored on disk
2025-11-27T14:52:23.906844Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 65 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.906886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.906898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.906901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.906903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.906906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.906919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.906932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.907032Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.907450Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457653:3561] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 65
    Offset: 65
    WriteTimestampMS: 1764255143903
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.908187Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.908762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457657:3563], now have 1 active actors on pipe
2025-11-27T14:52:23.908854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.908868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.908906Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|3e834546-45c9d563-6612e0f8-a3263ff9_66 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.908921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.908925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.908929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.908931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.908937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.908955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.908967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.908970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.908983Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.908994Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.909080Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.909465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457657:3563] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|3e834546-45c9d563-6612e0f8-a3263ff9_66"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|3e834546-45c9d563-6612e0f8-a3263ff9_66"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 66
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.911844Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.912366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457660:3564], now have 1 active actors on pipe
2025-11-27T14:52:23.912833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.912854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.912878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 66 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.912898Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.912917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.912921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.912931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.912940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.912946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.912973Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 66 partNo 0
2025-11-27T14:52:23.913018Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 66 partNo 0 FormedBlobsCount 0 NewHead: Offset 66 PartNo 0 PackedSize 262214 count 1 nextOffset 67 batches 1
2025-11-27T14:52:23.913032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.913035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.913039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.913177Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 66,1 HeadOffset 66 endOffset 66 curOffset 67 d0000000000_00000000000000000066_00000_0000000001_00000? size 262204 WTime 1764255143912
2025-11-27T14:52:23.913226Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.913246Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 66 partNo 0 count 1 size 262204
2025-11-27T14:52:23.915752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 66 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.915785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.915809Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.915808Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 66 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.915818Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.915829Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 66, partNo: 0, Offset: 66 is stored on disk
2025-11-27T14:52:23.915864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.915875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.915878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.915880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.915883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.915892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.915906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.915988Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.916433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457660:3564] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 66
    Offset: 66
    WriteTimestampMS: 1764255143912
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.917249Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.917746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457664:3566], now have 1 active actors on pipe
2025-11-27T14:52:23.917856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.917868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.917902Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|fdebcfa1-93cd5f19-cbfb5b5-b395fa65_67 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.917916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.917920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.917923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.917927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.917930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.917936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.917940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.917943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.917955Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.917973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.918042Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.918319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457664:3566] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|fdebcfa1-93cd5f19-cbfb5b5-b395fa65_67"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|fdebcfa1-93cd5f19-cbfb5b5-b395fa65_67"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 67
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.920531Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.921011Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457667:3567], now have 1 active actors on pipe
2025-11-27T14:52:23.921485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.921500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.921537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 67 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.921554Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.921566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.921569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.921576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.921577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.921580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.921600Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 67 partNo 0
2025-11-27T14:52:23.921628Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 67 partNo 0 FormedBlobsCount 0 NewHead: Offset 67 PartNo 0 PackedSize 262214 count 1 nextOffset 68 batches 1
2025-11-27T14:52:23.921633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.921635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.921637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.921718Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 67,1 HeadOffset 67 endOffset 67 curOffset 68 d0000000000_00000000000000000067_00000_0000000001_00000? size 262204 WTime 1764255143921
2025-11-27T14:52:23.921964Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.921984Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 67 partNo 0 count 1 size 262204
2025-11-27T14:52:23.924262Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 67 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.924293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.924317Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.924325Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 67 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.924328Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.924338Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 67, partNo: 0, Offset: 67 is stored on disk
2025-11-27T14:52:23.924369Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.924380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.924383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.924385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.924387Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.924398Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.924410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.924539Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.924934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457667:3567] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 67
    Offset: 67
    WriteTimestampMS: 1764255143921
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.925715Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.926169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457671:3569], now have 1 active actors on pipe
2025-11-27T14:52:23.926258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.926274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.926310Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a21f264a-7ae864ed-821dd2e0-45bd5aa9_68 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.926325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.926328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.926333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.926337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.926340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.926349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.926352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.926356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.926369Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.926379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.926437Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.926708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457671:3569] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a21f264a-7ae864ed-821dd2e0-45bd5aa9_68"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a21f264a-7ae864ed-821dd2e0-45bd5aa9_68"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 68
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.928954Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.929415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457674:3570], now have 1 active actors on pipe
2025-11-27T14:52:23.929888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.929907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.929931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 68 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.929979Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.929997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.930000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.930011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.930013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.930018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.930040Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 68 partNo 0
2025-11-27T14:52:23.930076Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 68 partNo 0 FormedBlobsCount 0 NewHead: Offset 68 PartNo 0 PackedSize 262214 count 1 nextOffset 69 batches 1
2025-11-27T14:52:23.930090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.930093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.930096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.930214Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 68,1 HeadOffset 68 endOffset 68 curOffset 69 d0000000000_00000000000000000068_00000_0000000001_00000? size 262204 WTime 1764255143929
2025-11-27T14:52:23.930269Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.930286Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 68 partNo 0 count 1 size 262204
2025-11-27T14:52:23.932720Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 68 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.932745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.932768Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.932777Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.932786Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 68, partNo: 0, Offset: 68 is stored on disk
2025-11-27T14:52:23.932822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.932825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.932827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.932830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.932832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.932843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.932854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.932910Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 68 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.932999Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.933430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457674:3570] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 68
    Offset: 68
    WriteTimestampMS: 1764255143929
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.934340Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.934845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457678:3572], now have 1 active actors on pipe
2025-11-27T14:52:23.934942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.934958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.934994Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|53e34792-758c5211-d9e93670-a79bc144_69 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.935009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.935013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.935017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.935019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.935023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.935032Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.935035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.935038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.935050Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.935069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.935136Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.935375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457678:3572] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|53e34792-758c5211-d9e93670-a79bc144_69"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|53e34792-758c5211-d9e93670-a79bc144_69"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 69
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.937629Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.938192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457681:3573], now have 1 active actors on pipe
2025-11-27T14:52:23.938668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.938688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.938713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 69 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.938725Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.938749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.938757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.938775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.938779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.938784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.938825Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 69 partNo 0
2025-11-27T14:52:23.938880Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 69 partNo 0 FormedBlobsCount 0 NewHead: Offset 69 PartNo 0 PackedSize 262214 count 1 nextOffset 70 batches 1
2025-11-27T14:52:23.938895Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.938898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.938913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.939028Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 69,1 HeadOffset 69 endOffset 69 curOffset 70 d0000000000_00000000000000000069_00000_0000000001_00000? size 262204 WTime 1764255143938
2025-11-27T14:52:23.939086Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.939104Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 69 partNo 0 count 1 size 262204
2025-11-27T14:52:23.941854Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 69 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.941889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.941920Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.941941Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.941941Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 69 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.941951Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 69, partNo: 0, Offset: 69 is stored on disk
2025-11-27T14:52:23.942026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.942035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.942038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.942040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.942042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.942056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.942070Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.942181Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.942579Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457681:3573] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 69
    Offset: 69
    WriteTimestampMS: 1764255143938
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.943495Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.943968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457685:3575], now have 1 active actors on pipe
2025-11-27T14:52:23.944087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.944100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.944138Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f9a69d6d-32c68283-7359c427-643a173f_70 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.944153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.944162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.944179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.944186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.944190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.944200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.944203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.944207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.944224Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.944242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.944294Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.944535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457685:3575] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f9a69d6d-32c68283-7359c427-643a173f_70"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f9a69d6d-32c68283-7359c427-643a173f_70"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 70
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.946948Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.947383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457689:3577], now have 1 active actors on pipe
2025-11-27T14:52:23.947834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.947850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.947877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 70 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.947895Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.947915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.947925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.947936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.947945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.947950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.947989Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 70 partNo 0
2025-11-27T14:52:23.948026Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 70 partNo 0 FormedBlobsCount 0 NewHead: Offset 70 PartNo 0 PackedSize 262214 count 1 nextOffset 71 batches 1
2025-11-27T14:52:23.948039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.948048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.948051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.948188Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 70,1 HeadOffset 70 endOffset 70 curOffset 71 d0000000000_00000000000000000070_00000_0000000001_00000? size 262204 WTime 1764255143947
2025-11-27T14:52:23.948231Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.948250Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 70 partNo 0 count 1 size 262204
2025-11-27T14:52:23.950546Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 70 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.950584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.950609Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.950625Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.950631Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 70 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.950635Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 70, partNo: 0, Offset: 70 is stored on disk
2025-11-27T14:52:23.950668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.950671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.950673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.950676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.950679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.950690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.950706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.951276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457689:3577] destroyed
CallPersQueueGRPC response:
2025-11-27T14:52:23.950826Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 70
    Offset: 70
    WriteTimestampMS: 1764255143947
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.952581Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.953124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457696:3582], now have 1 active actors on pipe
2025-11-27T14:52:23.953230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.953242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.953285Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a092a1d2-214ad39f-ffa58d57-a8052bb5_71 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.953301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.953306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.953312Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.953319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.953324Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.953336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.953340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.953343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.953363Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.953382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.953452Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.953748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457696:3582] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a092a1d2-214ad39f-ffa58d57-a8052bb5_71"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a092a1d2-214ad39f-ffa58d57-a8052bb5_71"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 71
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.956256Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.956770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457699:3583], now have 1 active actors on pipe
2025-11-27T14:52:23.957214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.957229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.957256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 71 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.957274Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.957290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.957300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.957314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.957321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.957326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.957362Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 71 partNo 0
2025-11-27T14:52:23.957404Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 71 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 262214 count 1 nextOffset 72 batches 1
2025-11-27T14:52:23.957418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.957421Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.957424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.957551Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 71,1 HeadOffset 71 endOffset 71 curOffset 72 d0000000000_00000000000000000071_00000_0000000001_00000? size 262204 WTime 1764255143956
2025-11-27T14:52:23.957594Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.957613Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 71 partNo 0 count 1 size 262204
2025-11-27T14:52:23.960225Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 71 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.960258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.960288Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.960287Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 71 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.960307Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.960317Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 71, partNo: 0, Offset: 71 is stored on disk
2025-11-27T14:52:23.960359Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.960367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.960370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.960372Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.960375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.960386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.960399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.960521Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 71
    Offset: 71
    WriteTimestampMS: 1764255143956
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-27T14:52:23.960983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457699:3583] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.961716Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.962186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457703:3585], now have 1 active actors on pipe
2025-11-27T14:52:23.962316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.962332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.962370Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|4d5baa67-c44d72f2-97b633ea-d8e156ca_72 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.962387Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.962397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.962408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.962411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.962416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.962427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.962430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.962433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.962449Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.962473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.962586Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.962852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457703:3585] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|4d5baa67-c44d72f2-97b633ea-d8e156ca_72"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|4d5baa67-c44d72f2-97b633ea-d8e156ca_72"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 72
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.965569Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.966124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457706:3586], now have 1 active actors on pipe
2025-11-27T14:52:23.966568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.966582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.966602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 72 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.966612Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.966625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.966628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.966639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.966648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.966652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.966678Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 72 partNo 0
2025-11-27T14:52:23.966722Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 72 partNo 0 FormedBlobsCount 0 NewHead: Offset 72 PartNo 0 PackedSize 262214 count 1 nextOffset 73 batches 1
2025-11-27T14:52:23.966737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.966739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.966743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.966871Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 72,1 HeadOffset 72 endOffset 72 curOffset 73 d0000000000_00000000000000000072_00000_0000000001_00000? size 262204 WTime 1764255143966
2025-11-27T14:52:23.966906Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.966920Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 72 partNo 0 count 1 size 262204
2025-11-27T14:52:23.969599Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 72 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.969628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.969656Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.969670Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.969682Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 72, partNo: 0, Offset: 72 is stored on disk
2025-11-27T14:52:23.969717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.969721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.969723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.969727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.969729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.969738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.969750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.969882Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 72 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.969967Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.970387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457706:3586] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 72
    Offset: 72
    WriteTimestampMS: 1764255143966
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.971148Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.971652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457710:3588], now have 1 active actors on pipe
2025-11-27T14:52:23.971746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.971755Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.971791Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|4b72f444-48962928-fd75e383-1856857e_73 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.971800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.971804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.971809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.971811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.971815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.971825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.971827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.971830Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.971847Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.971863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.971913Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.972208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457710:3588] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|4b72f444-48962928-fd75e383-1856857e_73"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|4b72f444-48962928-fd75e383-1856857e_73"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 73
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.974781Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.975660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457713:3589], now have 1 active actors on pipe
2025-11-27T14:52:23.975883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.975898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.975925Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 73 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.975942Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.975965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.975975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.975986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.975994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.975999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.976037Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 73 partNo 0
2025-11-27T14:52:23.976092Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 73 partNo 0 FormedBlobsCount 0 NewHead: Offset 73 PartNo 0 PackedSize 262214 count 1 nextOffset 74 batches 1
2025-11-27T14:52:23.976104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.976107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.976110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.976246Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 73,1 HeadOffset 73 endOffset 73 curOffset 74 d0000000000_00000000000000000073_00000_0000000001_00000? size 262204 WTime 1764255143976
2025-11-27T14:52:23.976298Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.976322Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 73 partNo 0 count 1 size 262204
2025-11-27T14:52:23.978667Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 73 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.978695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.978715Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 73 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.978719Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.978731Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.978747Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 73, partNo: 0, Offset: 73 is stored on disk
2025-11-27T14:52:23.978786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.978793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.978796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.978800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.978802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.978813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.978826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.978904Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.979283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457713:3589] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 73
    Offset: 73
    WriteTimestampMS: 1764255143975
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.980258Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.980734Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457717:3591], now have 1 active actors on pipe
2025-11-27T14:52:23.980835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.980847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.980886Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|5f450b98-cebb587b-fabc3ad7-65ea6e83_74 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.980902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.980906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.980911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.980918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.980937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.980953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.980961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.980964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.980979Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.980996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.981056Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.981292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457717:3591] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|5f450b98-cebb587b-fabc3ad7-65ea6e83_74"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|5f450b98-cebb587b-fabc3ad7-65ea6e83_74"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 74
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.984256Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457721:3593], now have 1 active actors on pipe
2025-11-27T14:52:23.984671Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.984683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.984708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 74 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.984721Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.984733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.984741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.984750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.984755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.984759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.984785Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 74 partNo 0
2025-11-27T14:52:23.984818Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 74 partNo 0 FormedBlobsCount 0 NewHead: Offset 74 PartNo 0 PackedSize 262214 count 1 nextOffset 75 batches 1
2025-11-27T14:52:23.984827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.984829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.984831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.984937Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 74,1 HeadOffset 74 endOffset 74 curOffset 75 d0000000000_00000000000000000074_00000_0000000001_00000? size 262204 WTime 1764255143984
2025-11-27T14:52:23.984970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-27T14:52:23.984980Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.984990Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 74 partNo 0 count 1 size 262204
2025-11-27T14:52:23.987291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 74 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.987319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.987338Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 74 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.987346Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.987358Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.987375Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 74, partNo: 0, Offset: 74 is stored on disk
2025-11-27T14:52:23.987415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.987424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.987426Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.987429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.987431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.987460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.983696Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.987473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.987573Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.987890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457721:3593] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 74
    Offset: 74
    WriteTimestampMS: 1764255143984
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.988757Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.989240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457725:3595], now have 1 active actors on pipe
2025-11-27T14:52:23.989319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.989333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.989370Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|e22efad3-922cc970-18f42dae-f2bc3903_75 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.989379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.989383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.989387Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.989389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.989393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.989404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.989407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.989411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.989426Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.989437Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.989518Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.989739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457725:3595] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|e22efad3-922cc970-18f42dae-f2bc3903_75"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|e22efad3-922cc970-18f42dae-f2bc3903_75"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 75
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:23.992073Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.992625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457728:3596], now have 1 active actors on pipe
2025-11-27T14:52:23.993052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.993071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.993096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 75 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:23.993116Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:23.993135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.993147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.993159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.993162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.993167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.993201Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 75 partNo 0
2025-11-27T14:52:23.993249Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 75 partNo 0 FormedBlobsCount 0 NewHead: Offset 75 PartNo 0 PackedSize 262214 count 1 nextOffset 76 batches 1
2025-11-27T14:52:23.993265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.993267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.993270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.993400Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 75,1 HeadOffset 75 endOffset 75 curOffset 76 d0000000000_00000000000000000075_00000_0000000001_00000? size 262204 WTime 1764255143992
2025-11-27T14:52:23.993449Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:23.993468Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 75 partNo 0 count 1 size 262204
2025-11-27T14:52:23.996123Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 75 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:23.996155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:23.996166Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 75 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:23.996185Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:23.996198Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:23.996208Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 75, partNo: 0, Offset: 75 is stored on disk
2025-11-27T14:52:23.996242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.996251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.996253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.996256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:23.996258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.996268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:23.996281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.996370Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:23.996747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457728:3596] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 75
    Offset: 75
    WriteTimestampMS: 1764255143992
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:23.997544Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:23.998046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418143001457732:3598], now have 1 active actors on pipe
2025-11-27T14:52:23.998165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:23.998182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:23.998224Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|74086673-4b872ab0-e2a61f67-87a1143f_76 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:23.998242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:23.998246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:23.998261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:23.998263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:23.998286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:23.998305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:23.998314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:23.998317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:23.998337Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:23.998359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:23.998452Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:23.998701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418143001457732:3598] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|74086673-4b872ab0-e2a61f67-87a1143f_76"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|74086673-4b872ab0-e2a61f67-87a1143f_76"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 76
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:24.001296Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.001757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425031:3599], now have 1 active actors on pipe
2025-11-27T14:52:24.002204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.002221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.002247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 76 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:24.002266Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:24.002287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.002296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.002309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.002316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.002321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.002357Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 76 partNo 0
2025-11-27T14:52:24.002403Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 76 partNo 0 FormedBlobsCount 0 NewHead: Offset 76 PartNo 0 PackedSize 262214 count 1 nextOffset 77 batches 1
2025-11-27T14:52:24.002416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.002420Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.002422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.002546Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 76,1 HeadOffset 76 endOffset 76 curOffset 77 d0000000000_00000000000000000076_00000_0000000001_00000? size 262204 WTime 1764255144002
2025-11-27T14:52:24.002595Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:24.002609Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 76 partNo 0 count 1 size 262204
2025-11-27T14:52:24.004946Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 76 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.004975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:24.004998Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 76 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.005003Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:24.005016Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:24.005032Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 76, partNo: 0, Offset: 76 is stored on disk
2025-11-27T14:52:24.005070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.005079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.005081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.005083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.005085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.005095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:24.005108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.005200Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.005535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425031:3599] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 76
    Offset: 76
    WriteTimestampMS: 1764255144002
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:24.006351Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.006888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425035:3601], now have 1 active actors on pipe
2025-11-27T14:52:24.006973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.006986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.007025Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|d6c7c032-b047bd23-93d232a2-cf916039_77 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:24.007039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.007045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.007051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.007053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.007061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.007075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.007084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.007086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.007105Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:24.007124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.007187Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.007451Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425035:3601] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|d6c7c032-b047bd23-93d232a2-cf916039_77"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|d6c7c032-b047bd23-93d232a2-cf916039_77"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 77
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:24.009445Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.009933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425038:3602], now have 1 active actors on pipe
2025-11-27T14:52:24.010377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.010396Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.010421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 77 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:24.010437Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:24.010457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.010466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.010476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.010483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.010488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.010523Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 77 partNo 0
2025-11-27T14:52:24.010566Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 77 partNo 0 FormedBlobsCount 0 NewHead: Offset 77 PartNo 0 PackedSize 262214 count 1 nextOffset 78 batches 1
2025-11-27T14:52:24.010580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.010584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.010587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.010735Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 77,1 HeadOffset 77 endOffset 77 curOffset 78 d0000000000_00000000000000000077_00000_0000000001_00000? size 262204 WTime 1764255144010
2025-11-27T14:52:24.010799Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:24.010819Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 77 partNo 0 count 1 size 262204
2025-11-27T14:52:24.013136Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 77 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.013161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:24.013177Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 77 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.013187Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:24.013198Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:24.013217Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 77, partNo: 0, Offset: 77 is stored on disk
2025-11-27T14:52:24.013259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.013267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.013270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.013272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.013274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.013283Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:24.013296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.013379Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.013700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425038:3602] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 77
    Offset: 77
    WriteTimestampMS: 1764255144010
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:24.014420Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.014894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425042:3604], now have 1 active actors on pipe
2025-11-27T14:52:24.015016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.015029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.015066Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|2320d883-da7eb63f-471f76ed-6014ee05_78 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:24.015079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.015084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.015093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.015095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.015099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.015110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.015116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.015118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.015130Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:24.015143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.015202Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.015417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425042:3604] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|2320d883-da7eb63f-471f76ed-6014ee05_78"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|2320d883-da7eb63f-471f76ed-6014ee05_78"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 78
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:24.017500Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.017902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425045:3605], now have 1 active actors on pipe
2025-11-27T14:52:24.018310Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.018323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.018343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 78 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:24.018353Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:24.018365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.018371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.018379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.018381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.018384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.018410Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 78 partNo 0
2025-11-27T14:52:24.018445Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 78 partNo 0 FormedBlobsCount 0 NewHead: Offset 78 PartNo 0 PackedSize 262214 count 1 nextOffset 79 batches 1
2025-11-27T14:52:24.018458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.018460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.018462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.018567Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 78,1 HeadOffset 78 endOffset 78 curOffset 79 d0000000000_00000000000000000078_00000_0000000001_00000? size 262204 WTime 1764255144018
2025-11-27T14:52:24.018607Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:24.018624Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 78 partNo 0 count 1 size 262204
2025-11-27T14:52:24.020947Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 78 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.020972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:24.020993Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:24.020994Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 78 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.021006Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:24.021018Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 78, partNo: 0, Offset: 78 is stored on disk
2025-11-27T14:52:24.021050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.021059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.021061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.021063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.021065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.021074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:24.021086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.021155Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.021457Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425045:3605] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 78
    Offset: 78
    WriteTimestampMS: 1764255144018
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:24.022246Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.022675Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425049:3607], now have 1 active actors on pipe
2025-11-27T14:52:24.022765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.022776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.022812Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|bb004eaa-19c41987-4ab8cc1d-7be5ad36_79 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:24.022825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.022829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.022834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.022837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.022840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.022851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.022862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.022864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.022879Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:24.022895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.022947Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:24.023197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425049:3607] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|bb004eaa-19c41987-4ab8cc1d-7be5ad36_79"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|bb004eaa-19c41987-4ab8cc1d-7be5ad36_79"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 79
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:24.025568Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.026073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425052:3608], now have 1 active actors on pipe
2025-11-27T14:52:24.026512Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.026533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.026558Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 79 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:24.026578Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:24.026595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.026605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.026630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.026639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.026645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.026678Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 79 partNo 0
2025-11-27T14:52:24.026729Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 79 partNo 0 FormedBlobsCount 0 NewHead: Offset 79 PartNo 0 PackedSize 262214 count 1 nextOffset 80 batches 1
2025-11-27T14:52:24.026745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.026749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.026751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.026884Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 79,1 HeadOffset 79 endOffset 79 curOffset 80 d0000000000_00000000000000000079_00000_0000000001_00000? size 262204 WTime 1764255144026
2025-11-27T14:52:24.026935Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:24.026953Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 79 partNo 0 count 1 size 262204
2025-11-27T14:52:24.029395Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 79 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.029430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:24.029458Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:24.029478Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:24.029488Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 79, partNo: 0, Offset: 79 is stored on disk
2025-11-27T14:52:24.029529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.029540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.029542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.029544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.029543Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 79 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.029546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.029557Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:24.029571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.029669Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.030047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425052:3608] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 79
    Offset: 79
    WriteTimestampMS: 1764255144026
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:24.030887Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.031352Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425056:3610], now have 1 active actors on pipe
2025-11-27T14:52:24.031513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.031528Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.031572Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|3dae5349-6d625f8a-66a1fcc3-6519d8db_80 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:24.031587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.031595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.031601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.031604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.031608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.031625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.031633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.031636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.031653Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:24.031673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.031743Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.031997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425056:3610] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|3dae5349-6d625f8a-66a1fcc3-6519d8db_80"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|3dae5349-6d625f8a-66a1fcc3-6519d8db_80"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 80
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:24.034250Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.034786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425059:3611], now have 1 active actors on pipe
2025-11-27T14:52:24.035221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.035238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.035262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 80 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:24.035278Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:24.035295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.035304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.035314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.035321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.035326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.035361Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 80 partNo 0
2025-11-27T14:52:24.035418Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 80 partNo 0 FormedBlobsCount 0 NewHead: Offset 80 PartNo 0 PackedSize 262214 count 1 nextOffset 81 batches 1
2025-11-27T14:52:24.035449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.035453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.035461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.035581Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 80,1 HeadOffset 80 endOffset 80 curOffset 81 d0000000000_00000000000000000080_00000_0000000001_00000? size 262204 WTime 1764255144035
2025-11-27T14:52:24.035626Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:24.035642Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 80 partNo 0 count 1 size 262204
2025-11-27T14:52:24.038024Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 80 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.038049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:24.038073Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:24.038080Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 80 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.038089Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:24.038098Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 80, partNo: 0, Offset: 80 is stored on disk
2025-11-27T14:52:24.038133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.038140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.038143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.038146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.038147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.038158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:24.038169Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.038260Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 80
    Offset: 80
    WriteTimestampMS: 1764255144034
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-27T14:52:24.038673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425059:3611] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:24.039369Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.039810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425063:3613], now have 1 active actors on pipe
2025-11-27T14:52:24.039898Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.039911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.039944Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|4cb82436-643f2a12-b871d3be-8d5c7b14_81 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:24.039957Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.039966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.039970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.039973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.039977Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.039991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.039998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.040004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.040022Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:24.040039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.040110Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.040321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425063:3613] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|4cb82436-643f2a12-b871d3be-8d5c7b14_81"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|4cb82436-643f2a12-b871d3be-8d5c7b14_81"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 81
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:24.042531Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.043007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425066:3614], now have 1 active actors on pipe
2025-11-27T14:52:24.043423Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.043456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.043481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 81 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:24.043494Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:24.043513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.043523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.043534Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.043541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.043546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.043581Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 81 partNo 0
2025-11-27T14:52:24.043625Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 81 partNo 0 FormedBlobsCount 0 NewHead: Offset 81 PartNo 0 PackedSize 262214 count 1 nextOffset 82 batches 1
2025-11-27T14:52:24.043640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.043643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.043646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.043781Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 81,1 HeadOffset 81 endOffset 81 curOffset 82 d0000000000_00000000000000000081_00000_0000000001_00000? size 262204 WTime 1764255144043
2025-11-27T14:52:24.043828Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:24.043847Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 81 partNo 0 count 1 size 262204
2025-11-27T14:52:24.046324Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 81 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.046354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:24.046375Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 81 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.046381Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:24.046392Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:24.046407Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 81, partNo: 0, Offset: 81 is stored on disk
2025-11-27T14:52:24.046449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.046457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.046459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.046464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.046467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.046476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:24.046488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.046583Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.046903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425066:3614] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 81
    Offset: 81
    WriteTimestampMS: 1764255144043
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:24.047777Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.048192Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425070:3616], now have 1 active actors on pipe
2025-11-27T14:52:24.048296Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.048306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.048337Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|8e08677c-d98f457e-46979edb-a18a09a8_82 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:24.048349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.048353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.048356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.048358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.048362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.048370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.048377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.048380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.048393Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:24.048408Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.048478Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|8e08677c-d98f457e-46979edb-a18a09a8_82"
    Status: Active
    SeqNo: 0
  }
}
2025-11-27T14:52:24.048820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425070:3616] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|8e08677c-d98f457e-46979edb-a18a09a8_82"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 82
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:24.051000Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.051422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425073:3617], now have 1 active actors on pipe
2025-11-27T14:52:24.051877Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.051896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.051920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 82 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:24.051931Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:24.051945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.051953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.051964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.051971Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.051975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.052008Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 82 partNo 0
2025-11-27T14:52:24.052050Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 82 partNo 0 FormedBlobsCount 0 NewHead: Offset 82 PartNo 0 PackedSize 262214 count 1 nextOffset 83 batches 1
2025-11-27T14:52:24.052075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.052079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.052082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.052197Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 82,1 HeadOffset 82 endOffset 82 curOffset 83 d0000000000_00000000000000000082_00000_0000000001_00000? size 262204 WTime 1764255144051
2025-11-27T14:52:24.052240Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:24.052257Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 82 partNo 0 count 1 size 262204
2025-11-27T14:52:24.054866Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 82 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.054894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:24.054921Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:24.054924Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 82 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.054931Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:24.054940Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 82, partNo: 0, Offset: 82 is stored on disk
2025-11-27T14:52:24.054967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.054997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.054999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.055001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.055003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.055010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:24.055020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.055120Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.055449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425073:3617] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 82
    Offset: 82
    WriteTimestampMS: 1764255144051
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:24.056336Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.056749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425077:3619], now have 1 active actors on pipe
2025-11-27T14:52:24.056851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.056862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.056897Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|bc08df10-d8c2d042-8396dc8-2bfbcce7_83 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:24.056909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.056912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.056918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.056924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.056929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.056938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.056945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.056947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.056959Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:24.056975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.057033Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.057252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425077:3619] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|bc08df10-d8c2d042-8396dc8-2bfbcce7_83"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|bc08df10-d8c2d042-8396dc8-2bfbcce7_83"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 83
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:24.059480Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.059887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425080:3620], now have 1 active actors on pipe
2025-11-27T14:52:24.060305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.060321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.060344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 83 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:24.060355Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:24.060372Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.060375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.060385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.060392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.060396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.060426Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 83 partNo 0
2025-11-27T14:52:24.060469Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 83 partNo 0 FormedBlobsCount 0 NewHead: Offset 83 PartNo 0 PackedSize 262214 count 1 nextOffset 84 batches 1
2025-11-27T14:52:24.060497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.060500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.060502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.060616Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 83,1 HeadOffset 83 endOffset 83 curOffset 84 d0000000000_00000000000000000083_00000_0000000001_00000? size 262204 WTime 1764255144060
2025-11-27T14:52:24.060656Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:24.060672Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 83 partNo 0 count 1 size 262204
2025-11-27T14:52:24.063009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 83 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.063035Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:24.063058Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:24.063062Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 83 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.063068Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:24.063079Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 83, partNo: 0, Offset: 83 is stored on disk
2025-11-27T14:52:24.063108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.063116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.063118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.063121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.063123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.063133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:24.063145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.063230Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.063570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425080:3620] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 83
    Offset: 83
    WriteTimestampMS: 1764255144060
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:24.064472Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.064916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425084:3622], now have 1 active actors on pipe
2025-11-27T14:52:24.065023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.065035Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.065072Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|58e320c9-762bbbf2-b571212b-d12aa6c8_84 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:24.065085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.065089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.065094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.065100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.065104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.065117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.065124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.065128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.065142Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:24.065174Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.065238Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.065513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425084:3622] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|58e320c9-762bbbf2-b571212b-d12aa6c8_84"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|58e320c9-762bbbf2-b571212b-d12aa6c8_84"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 84
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:24.067977Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.068441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425087:3623], now have 1 active actors on pipe
2025-11-27T14:52:24.068872Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.068888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.068912Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 84 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:24.068927Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:24.068943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.068946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.068956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.068961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.068968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.068998Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 84 partNo 0
2025-11-27T14:52:24.069036Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 84 partNo 0 FormedBlobsCount 0 NewHead: Offset 84 PartNo 0 PackedSize 262214 count 1 nextOffset 85 batches 1
2025-11-27T14:52:24.069046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.069048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.069050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.069145Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 84,1 HeadOffset 84 endOffset 84 curOffset 85 d0000000000_00000000000000000084_00000_0000000001_00000? size 262204 WTime 1764255144068
2025-11-27T14:52:24.069176Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:24.069194Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 84 partNo 0 count 1 size 262204
2025-11-27T14:52:24.071829Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 84 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.071862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:24.071880Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 84 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.071891Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:24.071903Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:24.071917Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 84, partNo: 0, Offset: 84 is stored on disk
2025-11-27T14:52:24.071972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.071982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.071984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.071986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.071988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.072000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:24.072015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.072092Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.072391Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425087:3623] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 84
    Offset: 84
    WriteTimestampMS: 1764255144068
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:24.073282Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.073742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425091:3625], now have 1 active actors on pipe
2025-11-27T14:52:24.073826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.073839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.073878Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|cf3cd2f-3026c270-ea154a3b-210a0513_85 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:24.073892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.073899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.073904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.073909Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.073913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.073923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.073931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.073933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.073948Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:24.073965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.074014Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.074197Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425091:3625] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|cf3cd2f-3026c270-ea154a3b-210a0513_85"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|cf3cd2f-3026c270-ea154a3b-210a0513_85"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 85
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:24.076075Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.076485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425094:3626], now have 1 active actors on pipe
2025-11-27T14:52:24.076843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.076862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.076885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 85 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:24.076898Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:24.076917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.076925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.076936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.076943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.076947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.076980Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 85 partNo 0
2025-11-27T14:52:24.077022Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 85 partNo 0 FormedBlobsCount 0 NewHead: Offset 85 PartNo 0 PackedSize 262214 count 1 nextOffset 86 batches 1
2025-11-27T14:52:24.077036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.077043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.077048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.077180Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 85,1 HeadOffset 85 endOffset 85 curOffset 86 d0000000000_00000000000000000085_00000_0000000001_00000? size 262204 WTime 1764255144076
2025-11-27T14:52:24.077230Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:24.077249Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 85 partNo 0 count 1 size 262204
2025-11-27T14:52:24.079451Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 85 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.079479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:24.079502Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:24.079517Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:24.079521Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 85 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.079529Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 85, partNo: 0, Offset: 85 is stored on disk
2025-11-27T14:52:24.079562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.079569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.079572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.079574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.079578Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.079587Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:24.079613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.079704Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:24.080032Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425094:3626] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 85
    Offset: 85
    WriteTimestampMS: 1764255144076
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:24.080728Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.081146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425098:3628], now have 1 active actors on pipe
2025-11-27T14:52:24.081235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.081247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.081280Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b7396cfe-34af448d-48955dee-9470d9d9_86 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:24.081293Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.081296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.081311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.081318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.081321Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.081332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.081339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.081341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.081353Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:24.081365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.081434Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:24.081652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425098:3628] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b7396cfe-34af448d-48955dee-9470d9d9_86"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b7396cfe-34af448d-48955dee-9470d9d9_86"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 86
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:24.083701Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.084150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425101:3629], now have 1 active actors on pipe
2025-11-27T14:52:24.084527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.084541Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.084563Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 86 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:24.084575Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:24.084597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.084606Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.084616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.084618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.084625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.084654Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 86 partNo 0
2025-11-27T14:52:24.084695Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 86 partNo 0 FormedBlobsCount 0 NewHead: Offset 86 PartNo 0 PackedSize 262214 count 1 nextOffset 87 batches 1
2025-11-27T14:52:24.084708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.084711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.084713Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.084837Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 86,1 HeadOffset 86 endOffset 86 curOffset 87 d0000000000_00000000000000000086_00000_0000000001_00000? size 262204 WTime 1764255144084
2025-11-27T14:52:24.084880Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:24.084898Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 86 partNo 0 count 1 size 262204
2025-11-27T14:52:24.085110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2281: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-27T14:52:24.086986Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 86 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.087016Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:24.087038Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:24.087045Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 86 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.087052Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:24.087059Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 86, partNo: 0, Offset: 86 is stored on disk
2025-11-27T14:52:24.087080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.087088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.087089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.087091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.087092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.087099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:24.087107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.087191Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.087426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425101:3629] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 86
    Offset: 86
    WriteTimestampMS: 1764255144084
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:24.088163Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.088556Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425105:3631], now have 1 active actors on pipe
2025-11-27T14:52:24.088639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.088651Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.088677Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|87a2f7cd-e36b8e86-233f55fc-1df0ec19_87 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:24.088683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.088687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.088690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.088692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.088697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.088706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.088709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.088711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.088723Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:24.088733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.088784Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.088944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425105:3631] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|87a2f7cd-e36b8e86-233f55fc-1df0ec19_87"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|87a2f7cd-e36b8e86-233f55fc-1df0ec19_87"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 87
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:24.090787Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.091170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425108:3632], now have 1 active actors on pipe
2025-11-27T14:52:24.091520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.091538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.091562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 87 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:24.091580Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:24.091598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.091608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.091617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.091623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.091627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.091658Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 87 partNo 0
2025-11-27T14:52:24.091697Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 87 partNo 0 FormedBlobsCount 0 NewHead: Offset 87 PartNo 0 PackedSize 262214 count 1 nextOffset 88 batches 1
2025-11-27T14:52:24.091712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.091715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.091718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.091842Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 87,1 HeadOffset 87 endOffset 87 curOffset 88 d0000000000_00000000000000000087_00000_0000000001_00000? size 262204 WTime 1764255144091
2025-11-27T14:52:24.091891Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:24.091910Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 87 partNo 0 count 1 size 262204
2025-11-27T14:52:24.093845Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 87 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.093865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:24.093883Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:24.093892Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:24.093892Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 87 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.093897Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 87, partNo: 0, Offset: 87 is stored on disk
2025-11-27T14:52:24.093921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.093923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.093925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.093926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.093928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.093934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:24.093943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.094034Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:24.094357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425108:3632] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 87
    Offset: 87
    WriteTimestampMS: 1764255144091
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:24.095112Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.095455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425112:3634], now have 1 active actors on pipe
2025-11-27T14:52:24.095554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.095566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.095602Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|129181a5-e3dda20f-e400644-d2415bf5_88 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:24.095617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.095626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.095631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.095633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.095636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.095646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.095649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.095653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.095673Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:24.095689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.095740Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:24.095943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425112:3634] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|129181a5-e3dda20f-e400644-d2415bf5_88"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|129181a5-e3dda20f-e400644-d2415bf5_88"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 88
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:24.097488Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.097805Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425115:3635], now have 1 active actors on pipe
2025-11-27T14:52:24.098132Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.098144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.098163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 88 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:24.098175Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:24.098185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.098191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.098198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.098200Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.098203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.098225Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 88 partNo 0
2025-11-27T14:52:24.098264Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 88 partNo 0 FormedBlobsCount 0 NewHead: Offset 88 PartNo 0 PackedSize 262214 count 1 nextOffset 89 batches 1
2025-11-27T14:52:24.098273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.098275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.098279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.098361Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 88,1 HeadOffset 88 endOffset 88 curOffset 89 d0000000000_00000000000000000088_00000_0000000001_00000? size 262204 WTime 1764255144097
2025-11-27T14:52:24.098394Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:24.098406Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 88 partNo 0 count 1 size 262204
2025-11-27T14:52:24.100000Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 88 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.100022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:24.100038Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 88 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.100040Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:24.100047Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:24.100053Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 88, partNo: 0, Offset: 88 is stored on disk
2025-11-27T14:52:24.100088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.100093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.100095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.100096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.100097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.100103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:24.100110Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.100169Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.100397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425115:3635] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 88
    Offset: 88
    WriteTimestampMS: 1764255144097
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:24.101022Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.101355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425119:3637], now have 1 active actors on pipe
2025-11-27T14:52:24.101444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.101455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.101487Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|873a0cc2-89b2a27a-43e4e314-d084f04_89 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:24.101498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.101501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.101505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.101507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.101510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.101519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.101521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.101524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.101535Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:24.101549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.101597Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.101791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425119:3637] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|873a0cc2-89b2a27a-43e4e314-d084f04_89"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|873a0cc2-89b2a27a-43e4e314-d084f04_89"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 89
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:24.103612Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.103916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425122:3638], now have 1 active actors on pipe
2025-11-27T14:52:24.104248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.104257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.104274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 89 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:24.104283Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:24.104295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.104299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.104305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.104311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.104313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.104335Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 89 partNo 0
2025-11-27T14:52:24.104365Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 89 partNo 0 FormedBlobsCount 0 NewHead: Offset 89 PartNo 0 PackedSize 262214 count 1 nextOffset 90 batches 1
2025-11-27T14:52:24.104373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.104374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.104376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.104467Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 89,1 HeadOffset 89 endOffset 89 curOffset 90 d0000000000_00000000000000000089_00000_0000000001_00000? size 262204 WTime 1764255144104
2025-11-27T14:52:24.104498Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:24.104510Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 89 partNo 0 count 1 size 262204
2025-11-27T14:52:24.106248Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 89 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.106272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:24.106284Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 89 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.106290Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:24.106297Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:24.106303Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 89, partNo: 0, Offset: 89 is stored on disk
2025-11-27T14:52:24.106328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.106335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.106336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.106338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.106339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.106345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:24.106353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.106456Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:24.106747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425122:3638] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 89
    Offset: 89
    WriteTimestampMS: 1764255144104
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:24.107316Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.107673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425126:3640], now have 1 active actors on pipe
2025-11-27T14:52:24.107746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.107758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.107792Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|1c5f77c2-97ec910b-58f82aee-ac54ea7d_90 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:24.107805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.107809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.107815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.107821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.107825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.107833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.107837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.107839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.107851Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:24.107867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.107914Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|1c5f77c2-97ec910b-58f82aee-ac54ea7d_90"
    Status: Active
    SeqNo: 0
  }
}
2025-11-27T14:52:24.108122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425126:3640] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|1c5f77c2-97ec910b-58f82aee-ac54ea7d_90"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 90
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:24.109903Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.110234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425129:3641], now have 1 active actors on pipe
2025-11-27T14:52:24.110561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.110574Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.110596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 90 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:24.110613Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:24.110635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.110642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.110652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.110658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.110663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.110691Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 90 partNo 0
2025-11-27T14:52:24.110728Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 90 partNo 0 FormedBlobsCount 0 NewHead: Offset 90 PartNo 0 PackedSize 262214 count 1 nextOffset 91 batches 1
2025-11-27T14:52:24.110741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.110745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.110748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.110862Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 90,1 HeadOffset 90 endOffset 90 curOffset 91 d0000000000_00000000000000000090_00000_0000000001_00000? size 262204 WTime 1764255144110
2025-11-27T14:52:24.110906Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:24.110924Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 90 partNo 0 count 1 size 262204
2025-11-27T14:52:24.112787Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 90 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.112816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:24.112838Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:24.112840Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 90 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.112851Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:24.112867Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 90, partNo: 0, Offset: 90 is stored on disk
2025-11-27T14:52:24.112902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.112911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.112913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.112915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.112917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.112925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:24.112937Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.113027Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:24.113370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425129:3641] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 90
    Offset: 90
    WriteTimestampMS: 1764255144110
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:24.113962Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.114297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425133:3643], now have 1 active actors on pipe
2025-11-27T14:52:24.114380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.114395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.114427Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|8660f1e-8528624c-d1d91646-59837901_91 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:24.114435Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.114444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.114449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.114451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.114454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.114463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.114466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.114468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.114481Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:24.114498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.114555Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|8660f1e-8528624c-d1d91646-59837901_91"
    Status: Active
    SeqNo: 0
  }
}
2025-11-27T14:52:24.114802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425133:3643] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|8660f1e-8528624c-d1d91646-59837901_91"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 91
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:24.116586Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.116968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425136:3644], now have 1 active actors on pipe
2025-11-27T14:52:24.117318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.117339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.117366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 91 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:24.117381Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:24.117393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.117396Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.117404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.117405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.117408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.117433Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 91 partNo 0
2025-11-27T14:52:24.117463Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 91 partNo 0 FormedBlobsCount 0 NewHead: Offset 91 PartNo 0 PackedSize 262214 count 1 nextOffset 92 batches 1
2025-11-27T14:52:24.117475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.117477Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.117479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.117575Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 91,1 HeadOffset 91 endOffset 91 curOffset 92 d0000000000_00000000000000000091_00000_0000000001_00000? size 262204 WTime 1764255144117
2025-11-27T14:52:24.117606Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:24.117619Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 91 partNo 0 count 1 size 262204
2025-11-27T14:52:24.119410Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 91 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.119453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:24.119476Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:24.119478Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 91 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.119491Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:24.119506Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 91, partNo: 0, Offset: 91 is stored on disk
2025-11-27T14:52:24.119540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.119547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.119549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.119551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.119552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.119561Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:24.119571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.119650Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.119980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425136:3644] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 91
    Offset: 91
    WriteTimestampMS: 1764255144117
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:24.120815Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.121790Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425141:3647], now have 1 active actors on pipe
2025-11-27T14:52:24.121827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.121835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.121879Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|549577cc-8bba9245-bee96009-ad474960_92 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:24.121897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.121901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.121906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.121908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.121912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.121925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.121931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.121934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.121952Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:24.121969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.122035Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.122248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425141:3647] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|549577cc-8bba9245-bee96009-ad474960_92"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|549577cc-8bba9245-bee96009-ad474960_92"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 92
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:24.124375Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.124815Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425144:3648], now have 1 active actors on pipe
2025-11-27T14:52:24.125259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.125278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.125303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 92 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:24.125316Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:24.125337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.125341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.125355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.125363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.125368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.125397Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 92 partNo 0
2025-11-27T14:52:24.125446Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 92 partNo 0 FormedBlobsCount 0 NewHead: Offset 92 PartNo 0 PackedSize 262214 count 1 nextOffset 93 batches 1
2025-11-27T14:52:24.125463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.125467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.125471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.125616Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 92,1 HeadOffset 92 endOffset 92 curOffset 93 d0000000000_00000000000000000092_00000_0000000001_00000? size 262204 WTime 1764255144125
2025-11-27T14:52:24.125666Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:24.125684Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 92 partNo 0 count 1 size 262204
2025-11-27T14:52:24.127772Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 92 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.127810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:24.127824Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 92 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.127833Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:24.127843Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:24.127854Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 92, partNo: 0, Offset: 92 is stored on disk
2025-11-27T14:52:24.127896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.127905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.127908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.127910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.127913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.127923Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:24.127936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.128000Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:24.128299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425144:3648] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 92
    Offset: 92
    WriteTimestampMS: 1764255144125
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:24.128950Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.129338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425148:3650], now have 1 active actors on pipe
2025-11-27T14:52:24.129400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.129416Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.129450Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|8a769e34-e3fdac96-8454bfa0-219549bd_93 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:24.129465Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.129468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.129473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.129475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.129479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.129487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.129490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.129493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.129505Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:24.129521Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.129616Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|8a769e34-e3fdac96-8454bfa0-219549bd_93"
    Status: Active
    SeqNo: 0
  }
}
2025-11-27T14:52:24.129851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425148:3650] destroyed
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|8a769e34-e3fdac96-8454bfa0-219549bd_93"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 93
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:24.131665Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.132129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425151:3651], now have 1 active actors on pipe
2025-11-27T14:52:24.132509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.132529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.132552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 93 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:24.132565Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:24.132578Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.132583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.132595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.132604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.132609Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.132635Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 93 partNo 0
2025-11-27T14:52:24.132673Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 93 partNo 0 FormedBlobsCount 0 NewHead: Offset 93 PartNo 0 PackedSize 262214 count 1 nextOffset 94 batches 1
2025-11-27T14:52:24.132698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.132705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.132708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.132848Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 93,1 HeadOffset 93 endOffset 93 curOffset 94 d0000000000_00000000000000000093_00000_0000000001_00000? size 262204 WTime 1764255144132
2025-11-27T14:52:24.132892Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:24.132908Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 93 partNo 0 count 1 size 262204
2025-11-27T14:52:24.135104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 93 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.135131Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:24.135139Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 93 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.135153Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:24.135162Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:24.135172Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 93, partNo: 0, Offset: 93 is stored on disk
2025-11-27T14:52:24.135213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.135222Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.135224Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.135227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.135230Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.135239Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:24.135251Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.135330Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.135650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425151:3651] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 93
    Offset: 93
    WriteTimestampMS: 1764255144132
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:24.136388Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.136825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425155:3653], now have 1 active actors on pipe
2025-11-27T14:52:24.136904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.136915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.136949Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|accd9045-e4875f52-e1963262-2278c2bb_94 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:24.136963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.136966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.136971Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.136973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.136976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.136985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.136987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.136991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.137004Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:24.137020Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.137058Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-27T14:52:24.137281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425155:3653] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|accd9045-e4875f52-e1963262-2278c2bb_94"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|accd9045-e4875f52-e1963262-2278c2bb_94"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 94
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:24.138894Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.139281Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425158:3654], now have 1 active actors on pipe
2025-11-27T14:52:24.139632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.139648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.139669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 94 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:24.139681Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:24.139700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.139708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.139718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.139724Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.139729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.139757Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 94 partNo 0
2025-11-27T14:52:24.139800Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 94 partNo 0 FormedBlobsCount 0 NewHead: Offset 94 PartNo 0 PackedSize 262214 count 1 nextOffset 95 batches 1
2025-11-27T14:52:24.139812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.139815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.139817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.139969Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 94,1 HeadOffset 94 endOffset 94 curOffset 95 d0000000000_00000000000000000094_00000_0000000001_00000? size 262204 WTime 1764255144139
2025-11-27T14:52:24.140026Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:24.140042Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 94 partNo 0 count 1 size 262204
2025-11-27T14:52:24.141980Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 94 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.142009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:24.142032Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:24.142033Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 94 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.142040Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:24.142049Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 94, partNo: 0, Offset: 94 is stored on disk
2025-11-27T14:52:24.142085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.142093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.142095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.142097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.142098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.142106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:24.142115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.142188Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.142443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425158:3654] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 94
    Offset: 94
    WriteTimestampMS: 1764255144139
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-27T14:52:24.143064Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.143466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425162:3656], now have 1 active actors on pipe
2025-11-27T14:52:24.143544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.143555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.143593Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b759fc74-cb6e0f95-55285e50-f323f763_95 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-27T14:52:24.143605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.143608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.143613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.143619Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.143622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.143630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.143633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.143635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.143645Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-27T14:52:24.143658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.143710Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.143903Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425162:3656] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b759fc74-cb6e0f95-55285e50-f323f763_95"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b759fc74-cb6e0f95-55285e50-f323f763_95"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 95
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:24.145724Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.146057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425165:3657], now have 1 active actors on pipe
2025-11-27T14:52:24.146499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.146513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.146532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2010: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 95 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-27T14:52:24.146541Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-27T14:52:24.146552Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.146555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2399: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-27T14:52:24.146563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-27T14:52:24.146566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.146570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2463: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-27T14:52:24.146592Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 95 partNo 0
2025-11-27T14:52:24.146626Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 95 partNo 0 FormedBlobsCount 0 NewHead: Offset 95 PartNo 0 PackedSize 262214 count 1 nextOffset 96 batches 1
2025-11-27T14:52:24.146639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-27T14:52:24.146642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2318: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-27T14:52:24.146644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.146770Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 95,1 HeadOffset 95 endOffset 95 curOffset 96 d0000000000_00000000000000000095_00000_0000000001_00000? size 262204 WTime 1764255144146
2025-11-27T14:52:24.146811Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:24.146827Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 95 partNo 0 count 1 size 262204
2025-11-27T14:52:24.148855Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 95 count 1 size 262204 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.148879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:24.148895Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:24.148899Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 95 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.148902Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-27T14:52:24.148909Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 95, partNo: 0, Offset: 95 is stored on disk
2025-11-27T14:52:24.148937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.148947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.148949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.148951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.148952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.148964Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000064_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.148966Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000065_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.148968Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000066_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.148971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000067_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.148974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000068_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.148976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000069_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.148978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000070_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.148980Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000071_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.148983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000072_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.148985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000073_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.148988Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000074_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.148990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000075_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.148993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000076_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.148995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000077_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.148998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000078_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.149001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000079_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.149003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000080_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.149006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000081_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.149009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000082_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.149011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000083_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.149014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000084_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.149017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000085_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.149020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000086_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.149022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000087_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.149024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000088_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.149026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000089_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.149029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000090_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.149032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000091_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.149034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000092_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.149036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000093_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.149038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000094_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.149040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000095_00000_0000000001_00000? (262204)
2025-11-27T14:52:24.149042Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 32 keys were taken away. Let's read 8390528 bytes
2025-11-27T14:52:24.149051Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.149064Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 32 blobs
2025-11-27T14:52:24.149066Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000064_00000_0000000001_00000?
2025-11-27T14:52:24.149075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000065_00000_0000000001_00000?
2025-11-27T14:52:24.149085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000066_00000_0000000001_00000?
2025-11-27T14:52:24.149101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000067_00000_0000000001_00000?
2025-11-27T14:52:24.149112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000068_00000_0000000001_00000?
2025-11-27T14:52:24.149132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000069_00000_0000000001_00000?
2025-11-27T14:52:24.149111Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.149142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000070_00000_0000000001_00000?
2025-11-27T14:52:24.149150Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000071_00000_0000000001_00000?
2025-11-27T14:52:24.149158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000072_00000_0000000001_00000?
2025-11-27T14:52:24.149179Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000073_00000_0000000001_00000?
2025-11-27T14:52:24.149191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000074_00000_0000000001_00000?
2025-11-27T14:52:24.149214Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000075_00000_0000000001_00000?
2025-11-27T14:52:24.149226Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000076_00000_0000000001_00000?
2025-11-27T14:52:24.149233Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000077_00000_0000000001_00000?
2025-11-27T14:52:24.149240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000078_00000_0000000001_00000?
2025-11-27T14:52:24.149247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000079_00000_0000000001_00000?
2025-11-27T14:52:24.149254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000080_00000_0000000001_00000?
CallPersQueueGRPC response:
2025-11-27T14:52:24.149295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000081_00000_0000000001_00000?
2025-11-27T14:52:24.149308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000082_00000_0000000001_00000?
2025-11-27T14:52:24.149315Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000083_00000_0000000001_00000?
2025-11-27T14:52:24.149322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000084_00000_0000000001_00000?
2025-11-27T14:52:24.149329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000085_00000_0000000001_00000?
2025-11-27T14:52:24.149335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000086_00000_0000000001_00000?
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 95
    Offset: 95
    WriteTimestampMS: 1764255144146
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-27T14:52:24.149350Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000087_00000_0000000001_00000?
2025-11-27T14:52:24.149362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000088_00000_0000000001_00000?
2025-11-27T14:52:24.149375Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000089_00000_0000000001_00000?
2025-11-27T14:52:24.149382Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000090_00000_0000000001_00000?
2025-11-27T14:52:24.149394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000091_00000_0000000001_00000?
CallPersQueueGRPC request to localhost:6400
2025-11-27T14:52:24.149401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000092_00000_0000000001_00000?
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdRead {
    ClientId: "user1"
    Offset: 0
    Count: 48
    ReadTimestampMs: 0
  }
}
2025-11-27T14:52:24.149413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000093_00000_0000000001_00000?
2025-11-27T14:52:24.149424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000094_00000_0000000001_00000?
2025-11-27T14:52:24.149432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000095_00000_0000000001_00000?
2025-11-27T14:52:24.149445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 32 blobs for compaction
2025-11-27T14:52:24.149454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425165:3657] destroyed
2025-11-27T14:52:24.149468Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 64 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:24.149476Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 65 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:24.149479Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 66 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:24.149483Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 67 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:24.149486Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 68 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:24.149489Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 69 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:24.149493Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 70 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:24.149495Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 71 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:24.149498Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 72 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:24.149501Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 73 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:24.149504Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 74 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:24.149507Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 75 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:24.149510Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 76 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:24.000000Z
2025-11-27T14:52:24.149512Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 77 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:24.000000Z
2025-11-27T14:52:24.149516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 78 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:24.000000Z
2025-11-27T14:52:24.149518Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 79 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:24.000000Z
2025-11-27T14:52:24.149520Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 80 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:24.000000Z
2025-11-27T14:52:24.149523Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 81 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:24.000000Z
2025-11-27T14:52:24.149526Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 82 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:24.000000Z
2025-11-27T14:52:24.149528Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 83 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:24.000000Z
2025-11-27T14:52:24.149531Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 84 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:24.000000Z
2025-11-27T14:52:24.149534Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 85 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:24.000000Z
2025-11-27T14:52:24.149537Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 86 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:24.000000Z
2025-11-27T14:52:24.149540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 87 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:24.000000Z
2025-11-27T14:52:24.149543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 88 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:24.000000Z
2025-11-27T14:52:24.149545Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 89 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:24.000000Z
2025-11-27T14:52:24.149548Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 90 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:24.000000Z
2025-11-27T14:52:24.149550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 91 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:24.000000Z
2025-11-27T14:52:24.149553Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 92 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:24.000000Z
2025-11-27T14:52:24.149556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 93 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:24.000000Z
2025-11-27T14:52:24.149558Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 94 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:24.000000Z
2025-11-27T14:52:24.149561Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 95 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-27T14:52:24.000000Z
2025-11-27T14:52:24.149576Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 32 blobs are from cache.
2025-11-27T14:52:24.149594Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 64 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149606Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 65 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149609Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 66 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149612Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 67 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149612Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-27T14:52:24.149615Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 68 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/32] d0000000000_00000000000000000064_00000_0000000001_00000?
2025-11-27T14:52:24.149619Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 69 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149621Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.149622Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 70 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149623Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000064_00000_0000000001_00000?, parameters.CurOffset=64
2025-11-27T14:52:24.149625Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 71 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149629Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 72 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149635Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 73 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149639Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 74 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149642Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 75 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149645Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 76 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149647Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 77 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149650Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 78 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149653Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 79 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149656Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 80 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149659Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 81 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149661Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 82 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149664Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 83 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149666Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 84 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149669Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 85 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149672Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 86 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149675Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 87 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149677Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 88 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149680Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 89 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149682Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 90 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149685Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 91 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149688Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 92 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149691Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 93 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149694Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 94 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149696Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 95 partno 0 count 1 parts 0 suffix '63'
2025-11-27T14:52:24.149721Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 64.0/1
2025-11-27T14:52:24.149733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 64 partNo 0
2025-11-27T14:52:24.149752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 64 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 262214 count 1 nextOffset 65 batches 1
2025-11-27T14:52:24.149760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 64:0 appended
2025-11-27T14:52:24.149764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/32] d0000000000_00000000000000000065_00000_0000000001_00000?
2025-11-27T14:52:24.149766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.149767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000065_00000_0000000001_00000?, parameters.CurOffset=65
2025-11-27T14:52:24.149834Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 65.0/1
2025-11-27T14:52:24.149843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 65 partNo 0
2025-11-27T14:52:24.149919Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.149995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 65 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 524387 count 2 nextOffset 66 batches 1
2025-11-27T14:52:24.150009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 65:0 appended
2025-11-27T14:52:24.150013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/32] d0000000000_00000000000000000066_00000_0000000001_00000?
2025-11-27T14:52:24.150014Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.150016Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000066_00000_0000000001_00000?, parameters.CurOffset=66
2025-11-27T14:52:24.150085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 66.0/1
2025-11-27T14:52:24.150094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 66 partNo 0
2025-11-27T14:52:24.150119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 66 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 786601 count 3 nextOffset 67 batches 2
2025-11-27T14:52:24.150126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 66:0 appended
2025-11-27T14:52:24.150129Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/32] d0000000000_00000000000000000067_00000_0000000001_00000?
2025-11-27T14:52:24.150130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.150132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000067_00000_0000000001_00000?, parameters.CurOffset=67
2025-11-27T14:52:24.150203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 67.0/1
2025-11-27T14:52:24.150211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 67 partNo 0
2025-11-27T14:52:24.150781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 67 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 1048774 count 4 nextOffset 68 batches 2
2025-11-27T14:52:24.150793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 67:0 appended
2025-11-27T14:52:24.150797Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/32] d0000000000_00000000000000000068_00000_0000000001_00000?
2025-11-27T14:52:24.150799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.150801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000068_00000_0000000001_00000?, parameters.CurOffset=68
2025-11-27T14:52:24.150869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 68.0/1
2025-11-27T14:52:24.150879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 68 partNo 0
2025-11-27T14:52:24.150888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 68 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 1310988 count 5 nextOffset 69 batches 3
2025-11-27T14:52:24.150890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 68:0 appended
2025-11-27T14:52:24.150894Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/32] d0000000000_00000000000000000069_00000_0000000001_00000?
2025-11-27T14:52:24.150896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.150898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000069_00000_0000000001_00000?, parameters.CurOffset=69
2025-11-27T14:52:24.150979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 69.0/1
2025-11-27T14:52:24.150989Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 69 partNo 0
2025-11-27T14:52:24.151120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 69 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 1573161 count 6 nextOffset 70 batches 3
2025-11-27T14:52:24.151130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 69:0 appended
2025-11-27T14:52:24.151134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/32] d0000000000_00000000000000000070_00000_0000000001_00000?
2025-11-27T14:52:24.151136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.151138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000070_00000_0000000001_00000?, parameters.CurOffset=70
2025-11-27T14:52:24.151195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 70.0/1
2025-11-27T14:52:24.151204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 70 partNo 0
2025-11-27T14:52:24.151211Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 70 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 1835375 count 7 nextOffset 71 batches 4
2025-11-27T14:52:24.151213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 70:0 appended
2025-11-27T14:52:24.151216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/32] d0000000000_00000000000000000071_00000_0000000001_00000?
2025-11-27T14:52:24.151217Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.151219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000071_00000_0000000001_00000?, parameters.CurOffset=71
2025-11-27T14:52:24.151282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 71.0/1
2025-11-27T14:52:24.151291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 71 partNo 0
2025-11-27T14:52:24.151407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 71 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 2097548 count 8 nextOffset 72 batches 4
2025-11-27T14:52:24.151417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 71:0 appended
2025-11-27T14:52:24.151420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[8/32] d0000000000_00000000000000000072_00000_0000000001_00000?
2025-11-27T14:52:24.151423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.151425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000072_00000_0000000001_00000?, parameters.CurOffset=72
2025-11-27T14:52:24.151509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 72.0/1
2025-11-27T14:52:24.151519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 72 partNo 0
2025-11-27T14:52:24.151527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 72 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 2359762 count 9 nextOffset 73 batches 5
2025-11-27T14:52:24.151529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 72:0 appended
2025-11-27T14:52:24.151532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[9/32] d0000000000_00000000000000000073_00000_0000000001_00000?
2025-11-27T14:52:24.151534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.151536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000073_00000_0000000001_00000?, parameters.CurOffset=73
2025-11-27T14:52:24.151597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 73.0/1
2025-11-27T14:52:24.151605Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 73 partNo 0
2025-11-27T14:52:24.151717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 73 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 2621935 count 10 nextOffset 74 batches 5
2025-11-27T14:52:24.151726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 73:0 appended
2025-11-27T14:52:24.151730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[10/32] d0000000000_00000000000000000074_00000_0000000001_00000?
2025-11-27T14:52:24.151732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.151734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000074_00000_0000000001_00000?, parameters.CurOffset=74
2025-11-27T14:52:24.151798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 74.0/1
2025-11-27T14:52:24.151807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 74 partNo 0
2025-11-27T14:52:24.151815Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 74 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 2884149 count 11 nextOffset 75 batches 6
2025-11-27T14:52:24.151817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 74:0 appended
2025-11-27T14:52:24.151821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[11/32] d0000000000_00000000000000000075_00000_0000000001_00000?
2025-11-27T14:52:24.151823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.151825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000075_00000_0000000001_00000?, parameters.CurOffset=75
2025-11-27T14:52:24.151892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 75.0/1
2025-11-27T14:52:24.151901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 75 partNo 0
2025-11-27T14:52:24.152011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 75 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 3146322 count 12 nextOffset 76 batches 6
2025-11-27T14:52:24.152019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 75:0 appended
2025-11-27T14:52:24.152023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[12/32] d0000000000_00000000000000000076_00000_0000000001_00000?
2025-11-27T14:52:24.152025Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.152027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000076_00000_0000000001_00000?, parameters.CurOffset=76
2025-11-27T14:52:24.152115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 76.0/1
2025-11-27T14:52:24.152122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 76 partNo 0
2025-11-27T14:52:24.152131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 76 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 3408536 count 13 nextOffset 77 batches 7
2025-11-27T14:52:24.152137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 76:0 appended
2025-11-27T14:52:24.152140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[13/32] d0000000000_00000000000000000077_00000_0000000001_00000?
2025-11-27T14:52:24.152142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.152144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000077_00000_0000000001_00000?, parameters.CurOffset=77
2025-11-27T14:52:24.152208Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 77.0/1
2025-11-27T14:52:24.152216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 77 partNo 0
2025-11-27T14:52:24.152327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 77 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 3670709 count 14 nextOffset 78 batches 7
2025-11-27T14:52:24.152336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 77:0 appended
2025-11-27T14:52:24.152340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[14/32] d0000000000_00000000000000000078_00000_0000000001_00000?
2025-11-27T14:52:24.152341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.152343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000078_00000_0000000001_00000?, parameters.CurOffset=78
2025-11-27T14:52:24.152413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 78.0/1
2025-11-27T14:52:24.152423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 78 partNo 0
2025-11-27T14:52:24.152431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 78 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 3932923 count 15 nextOffset 79 batches 8
2025-11-27T14:52:24.152438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 78:0 appended
2025-11-27T14:52:24.152441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[15/32] d0000000000_00000000000000000079_00000_0000000001_00000?
2025-11-27T14:52:24.152443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.152445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000079_00000_0000000001_00000?, parameters.CurOffset=79
2025-11-27T14:52:24.152499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 79.0/1
2025-11-27T14:52:24.152508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 79 partNo 0
2025-11-27T14:52:24.152620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 79 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 4195096 count 16 nextOffset 80 batches 8
2025-11-27T14:52:24.152628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 79:0 appended
2025-11-27T14:52:24.152632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[16/32] d0000000000_00000000000000000080_00000_0000000001_00000?
2025-11-27T14:52:24.152633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.152635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000080_00000_0000000001_00000?, parameters.CurOffset=80
2025-11-27T14:52:24.152704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 80.0/1
2025-11-27T14:52:24.152713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 80 partNo 0
2025-11-27T14:52:24.152722Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 80 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 4457310 count 17 nextOffset 81 batches 9
2025-11-27T14:52:24.152728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 80:0 appended
2025-11-27T14:52:24.152731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[17/32] d0000000000_00000000000000000081_00000_0000000001_00000?
2025-11-27T14:52:24.152732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.152735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000081_00000_0000000001_00000?, parameters.CurOffset=81
2025-11-27T14:52:24.152795Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 81.0/1
2025-11-27T14:52:24.152803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 81 partNo 0
2025-11-27T14:52:24.153299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 81 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 4719483 count 18 nextOffset 82 batches 9
2025-11-27T14:52:24.153311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 81:0 appended
2025-11-27T14:52:24.153314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[18/32] d0000000000_00000000000000000082_00000_0000000001_00000?
2025-11-27T14:52:24.153316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.153318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000082_00000_0000000001_00000?, parameters.CurOffset=82
2025-11-27T14:52:24.153384Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 82.0/1
2025-11-27T14:52:24.153394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 82 partNo 0
2025-11-27T14:52:24.153416Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 82 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 4981697 count 19 nextOffset 83 batches 10
2025-11-27T14:52:24.153422Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 82:0 appended
2025-11-27T14:52:24.153426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[19/32] d0000000000_00000000000000000083_00000_0000000001_00000?
2025-11-27T14:52:24.153427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.153430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000083_00000_0000000001_00000?, parameters.CurOffset=83
2025-11-27T14:52:24.153488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 83.0/1
2025-11-27T14:52:24.153496Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 83 partNo 0
2025-11-27T14:52:24.153979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 83 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 5243870 count 20 nextOffset 84 batches 10
2025-11-27T14:52:24.153990Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 83:0 appended
2025-11-27T14:52:24.153995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[20/32] d0000000000_00000000000000000084_00000_0000000001_00000?
2025-11-27T14:52:24.153998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.154001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000084_00000_0000000001_00000?, parameters.CurOffset=84
2025-11-27T14:52:24.154070Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 84.0/1
2025-11-27T14:52:24.154079Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 84 partNo 0
2025-11-27T14:52:24.154088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 84 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 5506084 count 21 nextOffset 85 batches 11
2025-11-27T14:52:24.154090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 84:0 appended
2025-11-27T14:52:24.154094Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[21/32] d0000000000_00000000000000000085_00000_0000000001_00000?
2025-11-27T14:52:24.154096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.154098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000085_00000_0000000001_00000?, parameters.CurOffset=85
2025-11-27T14:52:24.154158Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 85.0/1
2025-11-27T14:52:24.154168Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 85 partNo 0
2025-11-27T14:52:24.154654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 85 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 5768257 count 22 nextOffset 86 batches 11
2025-11-27T14:52:24.154665Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 85:0 appended
2025-11-27T14:52:24.154669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[22/32] d0000000000_00000000000000000086_00000_0000000001_00000?
2025-11-27T14:52:24.154671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.154674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000086_00000_0000000001_00000?, parameters.CurOffset=86
2025-11-27T14:52:24.154741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 86.0/1
2025-11-27T14:52:24.154751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 86 partNo 0
2025-11-27T14:52:24.154760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 86 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 6030471 count 23 nextOffset 87 batches 12
2025-11-27T14:52:24.154761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 86:0 appended
2025-11-27T14:52:24.154764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[23/32] d0000000000_00000000000000000087_00000_0000000001_00000?
2025-11-27T14:52:24.154766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.154768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000087_00000_0000000001_00000?, parameters.CurOffset=87
2025-11-27T14:52:24.154828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 87.0/1
2025-11-27T14:52:24.154837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 87 partNo 0
2025-11-27T14:52:24.155321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 87 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 6292644 count 24 nextOffset 88 batches 12
2025-11-27T14:52:24.155332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 87:0 appended
2025-11-27T14:52:24.155336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[24/32] d0000000000_00000000000000000088_00000_0000000001_00000?
2025-11-27T14:52:24.155338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.155341Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000088_00000_0000000001_00000?, parameters.CurOffset=88
2025-11-27T14:52:24.155414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 88.0/1
2025-11-27T14:52:24.155423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 88 partNo 0
2025-11-27T14:52:24.155447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 88 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 6554858 count 25 nextOffset 89 batches 13
2025-11-27T14:52:24.155450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 88:0 appended
2025-11-27T14:52:24.155453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[25/32] d0000000000_00000000000000000089_00000_0000000001_00000?
2025-11-27T14:52:24.155454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.155456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000089_00000_0000000001_00000?, parameters.CurOffset=89
2025-11-27T14:52:24.155519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 89.0/1
2025-11-27T14:52:24.155528Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 89 partNo 0
2025-11-27T14:52:24.156013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 89 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 6817031 count 26 nextOffset 90 batches 13
2025-11-27T14:52:24.156024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 89:0 appended
2025-11-27T14:52:24.156032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[26/32] d0000000000_00000000000000000090_00000_0000000001_00000?
2025-11-27T14:52:24.156033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.156036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000090_00000_0000000001_00000?, parameters.CurOffset=90
2025-11-27T14:52:24.156115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 90.0/1
2025-11-27T14:52:24.156125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 90 partNo 0
2025-11-27T14:52:24.156134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 90 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 7079245 count 27 nextOffset 91 batches 14
2025-11-27T14:52:24.156136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 90:0 appended
2025-11-27T14:52:24.156139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[27/32] d0000000000_00000000000000000091_00000_0000000001_00000?
2025-11-27T14:52:24.156141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.156143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000091_00000_0000000001_00000?, parameters.CurOffset=91
2025-11-27T14:52:24.156196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 91.0/1
2025-11-27T14:52:24.156206Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 91 partNo 0
2025-11-27T14:52:24.156698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 91 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 7341418 count 28 nextOffset 92 batches 14
2025-11-27T14:52:24.156710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 91:0 appended
2025-11-27T14:52:24.156715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[28/32] d0000000000_00000000000000000092_00000_0000000001_00000?
2025-11-27T14:52:24.156717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.156720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000092_00000_0000000001_00000?, parameters.CurOffset=92
2025-11-27T14:52:24.156788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 92.0/1
2025-11-27T14:52:24.156799Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 92 partNo 0
2025-11-27T14:52:24.156809Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 92 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 7603632 count 29 nextOffset 93 batches 15
2025-11-27T14:52:24.156811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 92:0 appended
2025-11-27T14:52:24.156814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[29/32] d0000000000_00000000000000000093_00000_0000000001_00000?
2025-11-27T14:52:24.156816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.156819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000093_00000_0000000001_00000?, parameters.CurOffset=93
2025-11-27T14:52:24.156872Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 93.0/1
2025-11-27T14:52:24.156881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 93 partNo 0
2025-11-27T14:52:24.163806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 93 partNo 0 result is x0000000000_00000000000000000062_00000_0000000031_00000 size 8128009
2025-11-27T14:52:24.163837Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1104: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000062_00000_0000000031_00000 new key d0000000000_00000000000000000062_00000_0000000031_00000 size 8128009 WTime 1764255144163
2025-11-27T14:52:24.163875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 93 partNo 0 FormedBlobsCount 1 NewHead: Offset 93 PartNo 0 PackedSize 262214 count 1 nextOffset 94 batches 1
2025-11-27T14:52:24.163883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 93:0 appended
2025-11-27T14:52:24.163891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[30/32] d0000000000_00000000000000000094_00000_0000000001_00000?
2025-11-27T14:52:24.163895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.163901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000094_00000_0000000001_00000?, parameters.CurOffset=94
2025-11-27T14:52:24.163983Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 94.0/1
2025-11-27T14:52:24.163998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 94 partNo 0
2025-11-27T14:52:24.164172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 94 partNo 0 FormedBlobsCount 0 NewHead: Offset 93 PartNo 0 PackedSize 524387 count 2 nextOffset 95 batches 1
2025-11-27T14:52:24.164185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 94:0 appended
2025-11-27T14:52:24.164191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[31/32] d0000000000_00000000000000000095_00000_0000000001_00000?
2025-11-27T14:52:24.164193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-27T14:52:24.164197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000095_00000_0000000001_00000?, parameters.CurOffset=95
2025-11-27T14:52:24.164254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 95.0/1
2025-11-27T14:52:24.164264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 95 partNo 0
2025-11-27T14:52:24.164277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 95 partNo 0 FormedBlobsCount 0 NewHead: Offset 93 PartNo 0 PackedSize 786601 count 3 nextOffset 96 batches 2
2025-11-27T14:52:24.164284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 95:0 appended
2025-11-27T14:52:24.164360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 93,3 HeadOffset 62 endOffset 64 curOffset 96 d0000000000_00000000000000000093_00000_0000000003_00000| size 786591 WTime 1764255144163
2025-11-27T14:52:24.164669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425170:3660], now have 1 active actors on pipe
2025-11-27T14:52:24.164713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.164724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.164742Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:24.164757Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 62 partNo 0 count 31 size 8128009
2025-11-27T14:52:24.164769Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 93 partNo 0 count 3 size 786591
2025-11-27T14:52:24.165211Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 4 Topic 'rt3.dc1--topic1' partition 0 user user1 offset 0 partno 0 count 48 size 26214400 endOffset 96 max time lag 0ms effective offset 0
2025-11-27T14:52:24.165413Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 4 added 2 blobs, size 16256018 count 63 last offset 31, current partition end offset: 96
2025-11-27T14:52:24.165428Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 4. Send blob request.
2025-11-27T14:52:24.165442Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 31 parts_count 0 source 1 size 8128009 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:24.165451Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 31 partno 0 count 31 parts_count 0 source 1 size 8128009 accessed 0 times before, last time 2025-11-27T14:52:23.000000Z
2025-11-27T14:52:24.165481Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 4. All 2 blobs are from cache.
2025-11-27T14:52:24.165510Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 31 parts 0 suffix '0'
2025-11-27T14:52:24.165519Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 2 blobs
2025-11-27T14:52:24.165523Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 31 parts 0 suffix '0'
2025-11-27T14:52:24.165728Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.166079Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 2 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.166430Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 4 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.166778Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 6 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.167123Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 8 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.167648Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 10 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.168193Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 12 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.168839Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 14 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.169376Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 16 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.169915Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 18 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.170456Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 20 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.170995Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 22 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.171560Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 24 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.172092Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 26 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.172721Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 28 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.172823Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 30 totakecount 31 count 1 size 262182 from pos 0 cbcount 1
2025-11-27T14:52:24.172931Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 31 totakecount 31 count 1 size 262182 from pos 0 cbcount 1
2025-11-27T14:52:24.173288Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 32 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.173637Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 34 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.173969Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 36 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.174303Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 38 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.174644Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 40 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.175001Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 42 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.175456Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 44 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.175999Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 46 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.176163Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.185559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.185578Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.185580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.185587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.185591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.188295Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 62 count 31 size 8128009 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.188319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 93 count 3 size 786591 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.188343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:24.188363Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 31 parts 0 suffix '0' size 8128009
2025-11-27T14:52:24.188373Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 2 parts 0 suffix '124' size 524387
2025-11-27T14:52:24.188385Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 64 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.188389Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 93 partno 0 count 3 parts 0 suffix '124' size 786591
2025-11-27T14:52:24.188559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.188573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.188576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.188578Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.188582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.188636Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:24.188652Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 64 size 262204 cause it's been evicted from L2. Actual L1 size: 36
2025-11-27T14:52:24.188655Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 62 size 524387 cause it's been evicted from L2. Actual L1 size: 35
2025-11-27T14:52:24.188661Z node 2 :PERSQUEUE NOTICE: read.h:372: [72075186224037892][PQCacheProxy]Have to remove new data from cache. Topic rt3.dc1--topic1, tablet id72075186224037892, cookie 0
2025-11-27T14:52:24.188668Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-27T14:52:24.188677Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000064_00000_0000000001_00000?(+) to d0000000000_00000000000000000064_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188679Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000065_00000_0000000001_00000?(+) to d0000000000_00000000000000000065_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188682Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000066_00000_0000000001_00000?(+) to d0000000000_00000000000000000066_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188684Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000067_00000_0000000001_00000?(+) to d0000000000_00000000000000000067_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188687Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000068_00000_0000000001_00000?(+) to d0000000000_00000000000000000068_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188689Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000069_00000_0000000001_00000?(+) to d0000000000_00000000000000000069_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188692Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000070_00000_0000000001_00000?(+) to d0000000000_00000000000000000070_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188695Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000071_00000_0000000001_00000?(+) to d0000000000_00000000000000000071_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188698Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000072_00000_0000000001_00000?(+) to d0000000000_00000000000000000072_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188700Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000073_00000_0000000001_00000?(+) to d0000000000_00000000000000000073_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188702Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000074_00000_0000000001_00000?(+) to d0000000000_00000000000000000074_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188705Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000075_00000_0000000001_00000?(+) to d0000000000_00000000000000000075_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188708Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000076_00000_0000000001_00000?(+) to d0000000000_00000000000000000076_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188709Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000077_00000_0000000001_00000?(+) to d0000000000_00000000000000000077_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188711Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000078_00000_0000000001_00000?(+) to d0000000000_00000000000000000078_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188713Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000079_00000_0000000001_00000?(+) to d0000000000_00000000000000000079_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188715Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000080_00000_0000000001_00000?(+) to d0000000000_00000000000000000080_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188717Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000081_00000_0000000001_00000?(+) to d0000000000_00000000000000000081_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188719Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000082_00000_0000000001_00000?(+) to d0000000000_00000000000000000082_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188721Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000083_00000_0000000001_00000?(+) to d0000000000_00000000000000000083_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188724Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000084_00000_0000000001_00000?(+) to d0000000000_00000000000000000084_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188729Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000085_00000_0000000001_00000?(+) to d0000000000_00000000000000000085_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188731Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000086_00000_0000000001_00000?(+) to d0000000000_00000000000000000086_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188734Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000087_00000_0000000001_00000?(+) to d0000000000_00000000000000000087_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188735Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000088_00000_0000000001_00000?(+) to d0000000000_00000000000000000088_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188737Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000089_00000_0000000001_00000?(+) to d0000000000_00000000000000000089_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188739Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000090_00000_0000000001_00000?(+) to d0000000000_00000000000000000090_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188741Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000091_00000_0000000001_00000?(+) to d0000000000_00000000000000000091_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188744Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000092_00000_0000000001_00000?(+) to d0000000000_00000000000000000092_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188746Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000093_00000_0000000001_00000?(+) to d0000000000_00000000000000000093_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188749Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000094_00000_0000000001_00000?(+) to d0000000000_00000000000000000094_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188751Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000095_00000_0000000001_00000?(+) to d0000000000_00000000000000000095_00000_0000000001_00000?(+)
2025-11-27T14:52:24.188753Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000062_00000_0000000002_00000|(+) to d0000000000_00000000000000000062_00000_0000000002_00000|(+)
2025-11-27T14:52:24.190088Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 65 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190104Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 66 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190109Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 67 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190116Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 68 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190122Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 69 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190126Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 70 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190130Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 71 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190134Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 72 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190140Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 73 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190144Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 74 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190148Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 75 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190152Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 76 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190156Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 77 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190161Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 78 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190165Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 79 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190168Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 80 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190174Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 81 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190184Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 82 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190188Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 83 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190193Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 84 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190197Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 85 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190200Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 86 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190204Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 87 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190207Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 88 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190211Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 89 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190214Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 90 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190218Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 91 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 92 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190230Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 93 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190235Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 94 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190238Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 95 count 1 actorID [2:7577418145187749200:2363]
2025-11-27T14:52:24.190258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2129: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-27T14:52:24.190259Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 65 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190264Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 66 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190268Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 67 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190271Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 68 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190275Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 69 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190279Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 70 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190283Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 71 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190283Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-27T14:52:24.190286Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 72 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190291Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 73 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.190295Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 74 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.190297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.190299Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 75 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.190301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.190309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-27T14:52:24.190313Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 76 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190317Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 77 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190321Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 78 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190326Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 79 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190329Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 80 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190333Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 81 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190339Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 82 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190343Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 83 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190346Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 84 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190351Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 85 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190354Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 86 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190357Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 87 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190360Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 88 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190363Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 89 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190365Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 90 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190368Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 91 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190372Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 92 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190374Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 93 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190377Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 94 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.190380Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 95 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-27T14:52:24.194778Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.206701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425170:3660] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdReadResult {
    MaxOffset: 96
    Result {
      Offset: 0
      Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
CallPersQueueGRPC request to localhost:6400
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdRead {
    ClientId: "user1"
    Offset: 0
    Count: 48
    ReadTimestampMs: 0
  }
}
2025-11-27T14:52:24.257476Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-27T14:52:24.257882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2720: [PQ: 72075186224037892] server connected, pipe [1:7577418147296425192:3673], now have 1 active actors on pipe
2025-11-27T14:52:24.257964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-27T14:52:24.257976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2634: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-27T14:52:24.258019Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 5 Topic 'rt3.dc1--topic1' partition 0 user user1 offset 0 partno 0 count 48 size 26214400 endOffset 96 max time lag 0ms effective offset 0
2025-11-27T14:52:24.258057Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 5 added 2 blobs, size 16256018 count 62 last offset 31, current partition end offset: 96
2025-11-27T14:52:24.258065Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 5. Send blob request.
2025-11-27T14:52:24.258078Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 31 parts_count 0 source 1 size 8128009 accessed 1 times before, last time 2025-11-27T14:52:24.000000Z
2025-11-27T14:52:24.258085Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 31 partno 0 count 31 parts_count 0 source 1 size 8128009 accessed 1 times before, last time 2025-11-27T14:52:24.000000Z
2025-11-27T14:52:24.258095Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 5. All 2 blobs are from cache.
2025-11-27T14:52:24.258124Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 31 parts 0 suffix '0'
2025-11-27T14:52:24.258132Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 2 blobs
2025-11-27T14:52:24.258137Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 31 parts 0 suffix '0'
2025-11-27T14:52:24.258575Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.258791Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 2 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.258978Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 4 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.259223Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 6 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.259597Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 8 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.259803Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 10 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.260188Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 12 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.260549Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 14 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.260955Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 16 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.261315Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 18 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.261635Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 20 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.261832Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 22 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.262033Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 24 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.262206Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 26 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.262621Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 28 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.262821Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 30 totakecount 31 count 1 size 262182 from pos 0 cbcount 1
2025-11-27T14:52:24.262910Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 31 totakecount 31 count 1 size 262182 from pos 0 cbcount 1
2025-11-27T14:52:24.263080Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 32 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.263265Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 34 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.263476Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 36 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.263664Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 38 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.263849Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 40 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.264035Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 42 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.264237Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 44 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.264425Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 46 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-27T14:52:24.264493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-27T14:52:24.274362Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-27T14:52:24.283750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2745: [PQ: 72075186224037892] server disconnected, pipe [1:7577418147296425192:3673] destroyed
2025-11-27T14:52:24.285842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.285861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.285865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.285868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.285872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdReadResult {
    MaxOffset: 96
    Result {
      Offset: 0
      Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-27T14:52:24.386275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.386304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.386306Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.386310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.386313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.486595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.486623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.486626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.486630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.486634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-27T14:52:24.586974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2305: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-27T14:52:24.587001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.587004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-27T14:52:24.587009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2364: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-27T14:52:24.587012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2323: [72075186224037892][Partition][0][StateIdle] Try persist
[[bad]]assertion failed at ydb/services/persqueue_v1/persqueue_ut.cpp:3344, virtual void NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TTestCaseCacheHead::Execute_(NUnitTest::TTestContext &): (info_half1.BlobsFromDisk == 1) failed: (0 != 1) [[rst]]
[[alt1]]TBackTrace::Capture()+28 (0x150BCE9C)
NUnitTest::NPrivate::RaiseError(char const*, TBasicString<char, std::__y1::char_traits<char>> const&, bool)+128 (0x15278EA0)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TTestCaseCacheHead::Execute_(NUnitTest::TTestContext&)+5702 (0x14DF1A66)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TCurrentTest::Execute()::'lambda'()::operator()() const+71 (0x14ED3D07)
NUnitTest::TTestBase::Run(std::__y1::function<void ()>, TBasicString<char, std::__y1::char_traits<char>> const&, char const*, bool)+120 (0x1527B078)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TCurrentTest::Execute()+436 (0x14ED3604)
NUnitTest::TTestFactory::Execute()+817 (0x1527B7C1)
NUnitTest::RunMain(int, char**)+3229 (0x1528DC2D)
??+0 (0x7F4366A9FD90)
__libc_start_main+128 (0x7F4366A9FE40)
_start+41 (0x13EDB029)
[[rst]]
