2025-11-14T16:02:56.239595Z node 1 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[1:7572612217082220165:2084];send_to=[0:7307199536658146131:7762515];
2025-11-14T16:02:56.239712Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
2025-11-14T16:02:56.244031Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:123: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, got undelivered to scheme cache: ActorUnknown
2025-11-14T16:02:56.253091Z node 1 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
test_client.cpp: SetPath # /home/runner/.ya/build/build_root/bku8/000129/r3tmp/tmptAhqXS/pdisk_1.dat
2025-11-14T16:02:56.258673Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-11-14T16:02:56.301013Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-11-14T16:02:56.301089Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=incorrect path status: LookupError;
2025-11-14T16:02:56.331562Z node 1 :IMPORT WARN: schemeshard_import.cpp:307: Table profiles were not loaded
2025-11-14T16:02:56.333822Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
TServer::EnableGrpc on GrpcPort 27509, node 1
2025-11-14T16:02:56.347983Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:190: distributable config is empty, broken or outdated, will use file: /home/runner/.ya/build/build_root/bku8/000129/r3tmp/yandex96y7yY.tmp
2025-11-14T16:02:56.348004Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:196: will try to initialize from file: /home/runner/.ya/build/build_root/bku8/000129/r3tmp/yandex96y7yY.tmp
2025-11-14T16:02:56.348111Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:202: successfully initialized from file: /home/runner/.ya/build/build_root/bku8/000129/r3tmp/yandex96y7yY.tmp
2025-11-14T16:02:56.348149Z node 1 :NET_CLASSIFIER ERROR: net_classifier.cpp:228: got bad distributable configuration
2025-11-14T16:02:56.350524Z INFO: TTestServer started on Port 14958 GrpcPort 27509
2025-11-14T16:02:56.359074Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-11-14T16:02:56.359100Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-11-14T16:02:56.363625Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Connecting -> Connected
2025-11-14T16:02:56.363739Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-11-14T16:02:56.363761Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-11-14T16:02:56.368235Z node 1 :HIVE WARN: hive_impl.cpp:811: HIVE#72057594037968897 Handle TEvInterconnect::TEvNodeConnected, NodeId 2 Cookie 2
2025-11-14T16:02:56.368874Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Connecting -> Connected
TClient is connected to server localhost:14958
PQClient connected to localhost:27509
WaitRootIsUp 'Root'...
TClient::Ls request: Root
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "Root"
    PathId: 1
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 1
    CreateStep: 0
    ParentPathId: 1
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 1
      SubDomainVersion: 0
      SecurityStateVersion: 0
    }
    ChildrenExist: false
  }
  Children {
    Name: ".sys"
    PathId: 18446744073709551615
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 0
    CreateStep: 0
    ParentPathId: 18446744073709551615
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 0
      Pl...
(TRUNCATED)
WaitRootIsUp 'Root' success.
2025-11-14T16:02:56.386809Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpAlterSubDomain, opId: 281474976720657:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_alter_subdomain.cpp:311)
waiting...
waiting...
2025-11-14T16:02:56.406181Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720659:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
waiting...
waiting...
waiting...
2025-11-14T16:02:56.571160Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-11-14T16:02:56.754940Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7572612216835583760:2300], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-14T16:02:56.754995Z node 2 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-14T16:02:56.755119Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7572612216835583779:2303], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-14T16:02:56.755134Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7572612216835583780:2304], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-14T16:02:56.755156Z node 2 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-14T16:02:56.756363Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateResourcePool, opId: 281474976715657:3, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:179)
2025-11-14T16:02:56.764393Z node 2 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [2:7572612216835583783:2305], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976715657 completed, doublechecking }
2025-11-14T16:02:56.855599Z node 2 :TX_PROXY ERROR: schemereq.cpp:593: Actor# [2:7572612216835583811:2180] txid# 281474976715658, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 9], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-11-14T16:02:56.861786Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720662:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
2025-11-14T16:02:56.862284Z node 1 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [1:7572612217082221224:2331], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-11-14T16:02:56.862422Z node 1 :KQP_SESSION WARN: kqp_session_actor.cpp:2517: SessionId: ydb://session/3?node_id=1&id=Y2YwN2UwODgtODhjNGM5N2ItZDQ3NmU3MGQtZDIyM2FhMDI=, ActorId: [1:7572612217082221182:2323], ActorState: ExecuteState, TraceId: 01ka1hp0mz6bqw1ekv6mje9tcy, ReplyQueryCompileError, status: SCHEME_ERROR, issues: { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } }, remove tx with tx_id: 
2025-11-14T16:02:56.862518Z node 2 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [2:7572612216835583818:2309], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-11-14T16:02:56.862665Z node 2 :KQP_SESSION WARN: kqp_session_actor.cpp:2517: SessionId: ydb://session/3?node_id=2&id=NTIyOWQxYTItYzEwYTM4YjktMzRmZjE4My05ZDkxOWE1Nw==, ActorId: [2:7572612216835583749:2298], ActorState: ExecuteState, TraceId: 01ka1hp0kh9t0erxvctpnrcaac, ReplyQueryCompileError, status: SCHEME_ERROR, issues: { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } }, remove tx with tx_id: 
2025-11-14T16:02:56.862854Z node 1 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-11-14T16:02:56.862862Z node 2 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-11-14T16:02:56.880624Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720663:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
2025-11-14T16:02:56.905423Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720664:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
=== Init DC: UPSERT INTO `/Root/PQ/Config/V2/Cluster` (name, balancer, local, enabled, weight) VALUES ("dc1", "localhost", true, true, 1000), ("dc2", "dc2.logbroker.yandex.net", false, true, 1000);
2025-11-14T16:02:56.954317Z node 1 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720665. Ctx: { TraceId: 01ka1hp0s55fjr7am5d1r4n17x, Database: , SessionId: ydb://session/3?node_id=1&id=ODZhYTdmMTMtYTY4YWVjZjYtZThjOWExZjktNzhlYTUyZDg=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
=== CheckClustersList. Subcribe to ClusterTracker from [1:7572612217082221660:3041] 
2025-11-14T16:02:57.241591Z node 1 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-11-14T16:02:57.255931Z node 2 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-11-14T16:03:01.240838Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[1:7572612217082220165:2084];send_to=[0:7307199536658146131:7762515];
2025-11-14T16:03:01.240922Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
=== CheckClustersList. Ok
PQ Client: create topic: rt3.dc1--topic1 with 1 partitions
CallPersQueueGRPC request to localhost:27509
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--topic1"
  }
}
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--topic1, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:27509
MetaRequest {
  CmdCreateTopic {
    Topic: "rt3.dc1--topic1"
    NumPartitions: 1
    Config {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
  }
}
2025-11-14T16:03:02.162098Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 129
ProxyErrorCode: 53
SchemeStatus: 1
FlatTxId {
  TxId: 281474976720671
  SchemeShardTabletId: 72057594046644480
  PathId: 13
}
ErrorCode: OK
AddTopic: rt3.dc1--topic1
===Run query:``DECLARE $version as Int64; DECLARE $path AS Utf8; DECLARE $cluster as Utf8; UPSERT INTO `/Root/PQ/Config/V2/Topics` (path, dc) VALUES ($path, $cluster); UPSERT INTO `/Root/PQ/Config/V2/Versions` (name, version) VALUES ("Topics", $version);`` with topic = topic1, dc = dc1
2025-11-14T16:03:02.179661Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1655: [72075186224037893][] pipe [1:7572612242852025856:3266] connected; active server actors: 1
2025-11-14T16:03:02.179914Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1519: [72075186224037893][rt3.dc1--topic1] updating configuration. Deleted partitions []. Added partitions [0]
2025-11-14T16:03:02.180086Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:971: [72075186224037893][rt3.dc1--topic1] Discovered subdomain [OwnerId: 72057594046644480, LocalPathId: 1] state, outOfSpace = 0 at RB 72075186224037893
2025-11-14T16:03:02.180586Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:140: [72075186224037893][rt3.dc1--topic1] BALANCER INIT DONE for rt3.dc1--topic1: (0, 72075186224037892) 
2025-11-14T16:03:02.182904Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2960: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-11-14T16:03:02.183529Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2992: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-11-14T16:03:02.183578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:787: [PQ: 72075186224037892] no config, start with empty partitions and default config
2025-11-14T16:03:02.183584Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:913: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-11-14T16:03:02.183598Z node 2 :PERSQUEUE INFO: pq_impl.cpp:612: [PQ: 72075186224037892] doesn't have tx writes info
2025-11-14T16:03:02.183680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852025881:2451], now have 1 active actors on pipe
2025-11-14T16:03:02.183684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852025855:3265], now have 1 active actors on pipe
2025-11-14T16:03:02.183875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3570: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-14T16:03:02.185351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-14T16:03:02.187405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852025917:3314], now have 1 active actors on pipe
2025-11-14T16:03:02.187601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3570: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-14T16:03:02.188758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-14T16:03:02.183731Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:404: [72075186224037893][rt3.dc1--topic1] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1
2025-11-14T16:03:02.188987Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitConfigStep
2025-11-14T16:03:02.189014Z node 2 :PERSQUEUE DEBUG: partition.cpp:1316: [72075186224037892][Partition][0][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-11-14T16:03:02.189070Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-11-14T16:03:02.189153Z node 2 :PERSQUEUE INFO: partition_init.cpp:1107: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7572612242605387869:2363]
2025-11-14T16:03:02.189949Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitFieldsStep
2025-11-14T16:03:02.189960Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--topic1:0:Initializer] Initializing completed.
2025-11-14T16:03:02.189964Z node 2 :PERSQUEUE INFO: partition.cpp:708: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--topic1' partition 0 generation 1 [2:7572612242605387869:2363]
2025-11-14T16:03:02.189971Z node 2 :PERSQUEUE DEBUG: partition.cpp:728: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--topic1 partitition 0 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-11-14T16:03:02.189978Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:107: [72075186224037892][Partition][0][StateIdle] Initializing MLP Consumers: 0
2025-11-14T16:03:02.190295Z node 2 :PERSQUEUE DEBUG: partition.cpp:4406: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 1
2025-11-14T16:03:02.190301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.190314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2400: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TTransaction[ProposeConfig])
2025-11-14T16:03:02.190324Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.190326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.190329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.190331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.190347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.190349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.190350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.190352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.190354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.190363Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:930: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-11-14T16:03:02.190370Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:70: [72075186224037892][Partition][0][StateIdle] Process MLP pending events. Count 0
2025-11-14T16:03:02.190431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3570: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-14T16:03:02.190459Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.192017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-14T16:03:02.192056Z node 2 :PERSQUEUE DEBUG: partition.cpp:1428: [72075186224037892][Partition][0][StateIdle] Handle TEvPQ::TEvTxCommit Step 1763136182234, TxId 281474976720671
2025-11-14T16:03:02.192068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.192071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.192073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.192079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2464: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TTransaction[ProposeConfig])
2025-11-14T16:03:02.192089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.192091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.192094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.192129Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.192860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.193004Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:490: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-11-14T16:03:02.193041Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:107: [72075186224037892][Partition][0][StateIdle] Initializing MLP Consumers: 0
2025-11-14T16:03:02.193072Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.193079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.193082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.193085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.193087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.193090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.193095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.193213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:398: [PQ: 72075186224037892] Apply new config PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 8388608 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--topic1" Version: 0 LocalDC: true Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } TopicPath: "/Root/PQ/rt3.dc1--topic1" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Version: 0 Generation: 0 } MonitoringProjectId: ""
2025-11-14T16:03:02.193232Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:913: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-11-14T16:03:02.193276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3570: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-14T16:03:02.196153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-14T16:03:02.196183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3570: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-14T16:03:02.196706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-14T16:03:02.196572Z node 1 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720672. Ctx: { TraceId: 01ka1hp5x029dn249cz3x70cmf, Database: , SessionId: ydb://session/3?node_id=1&id=MzljM2ZlNzAtMjBiNGQ5ZmMtYzY3MGVkOGMtZmJiNWQ4NjM=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
===Query complete
TClient::Ls request: /Root/PQ/rt3.dc1--topic1
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--topic1"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720671
    CreateStep: 1763136182234
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--topic1"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 600000...
(TRUNCATED)
GetTopicVersionFromPath:  record Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--topic1"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720671
    CreateStep: 1763136182234
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--topic1"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 8388608
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      YdbDatabasePath: "/Root"
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
    Partitions {
      PartitionId: 0
      TabletId: 72075186224037892
      Status: Active
    }
    AlterVersion: 1
    BalancerTabletID: 72075186224037893
    NextPartitionId: 1
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 1
      PlanResolution: 7
      Coordinators: 72057594046316545
      TimeCastBucketsPerMediator: 2
      Mediators: 72057594046382081
    }
    DomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    PathsInside: 12
    PathsLimit: 10000
    ShardsInside: 6
    ShardsLimit: 200000
    ResourcesDomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    DiskSpaceUsage {
      Tables {
        TotalSize: 0
        DataSize: 0
        IndexSize: 0
      }
      Topics {
        ReserveSize: 0
        AccountSize: 0
        DataSize: 0
        UsedReserveSize: 0
      }
    }
    PQPartitionsInside: 1
    PQPartitionsLimit: 1000000
    SchemeLimits {
      MaxDepth: 32
      MaxPaths: 10000
      MaxChildrenInDir: 100000
      MaxAclBytesSize: 10240
      MaxTableColumns: 200
      MaxTableColumnNameLength: 255
      MaxTableKeyColumns: 30
      MaxTableIndices: 20
      MaxShards: 200000
      MaxShardsInPath: 35000
      MaxConsistentCopyTargets: 10000
      MaxPathElementLength: 255
      ExtraPathSymbolsAllowed: "!\"#$%&\'()*+,-.:;<=>?@[\\]^_`{|}~"
      MaxPQPartitions: 1000000
      MaxTableCdcStreams: 5
      MaxExports: 10
      MaxImports: 10
      MaxColumnTableColumns: 10000
    }
  }
}
Path: "/Root/PQ/rt3.dc1--topic1"
 name rt3.dc1--topic1 version1
CallPersQueueGRPC request to localhost:27509
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--topic1"
  }
}
2025-11-14T16:03:02.206816Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
MetaResponse {
  CmdGetTopicMetadataResult {
    TopicInfo {
      Topic: "rt3.dc1--topic1"
      NumPartitions: 1
      Config {
        PartitionConfig {
          LifetimeSeconds: 86400
          LowWatermark: 8388608
          SourceIdLifetimeSeconds: 86400
          WriteSpeedInBytesPerSecond: 20000000
          BurstSize: 20000000
          SourceIdMaxCounts: 6000000
        }
        Version: 1
        LocalDC: true
        Codecs {
          Ids: 0
          Ids: 1
          Ids: 2
          Codecs: "raw"
          Codecs: "gzip"
          Codecs: "lzop"
        }
        TopicPath: "/Root/PQ/rt3.dc1--topic1"
        YdbDatabasePath: "/Root"
        Consumers {
          Name: "user"
          ReadFromTimestampsMs: 0
          FormatVersion: 0
          Version: 0
        }
      }
      ErrorCode: OK
    }
  }
}
=== Topic created, have version: 1
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.208760Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.209325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852025980:3355], now have 1 active actors on pipe
2025-11-14T16:03:02.209501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.209518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.209566Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|50fc2616-1c6602a7-326a67ee-a2c5a851_0 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.209582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.209590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.209596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.209598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.209604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.209611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.209614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.209617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.209632Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.209654Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.209755Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|50fc2616-1c6602a7-326a67ee-a2c5a851_0"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
2025-11-14T16:03:02.214679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852025980:3355] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|50fc2616-1c6602a7-326a67ee-a2c5a851_0"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 0
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.224404Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.224963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852025983:3356], now have 1 active actors on pipe
2025-11-14T16:03:02.227682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.227707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.227884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 0 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.228394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 0 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.228409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 0 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.228414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 0 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.228438Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.228457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.228461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.228470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.228473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.228478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.228480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.228485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.228514Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 0
2025-11-14T16:03:02.228528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.228532Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 1
2025-11-14T16:03:02.228536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.228539Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 2
2025-11-14T16:03:02.229740Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 0 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1048790 count 1 nextOffset 1 batches 3
2025-11-14T16:03:02.229756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.229760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.229764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.229840Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 0,1 HeadOffset 0 endOffset 0 curOffset 1 d0000000000_00000000000000000000_00000_0000000001_00002? size 1048780 WTime 1763136182229
2025-11-14T16:03:02.230957Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.230971Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.238512Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 0 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.238536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.238562Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.238569Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.238579Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 0, partNo: 0, Offset: 0 is stored on disk
2025-11-14T16:03:02.238584Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.238588Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 0, partNo: 1, Offset: 0 is stored on disk
2025-11-14T16:03:02.238594Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.238598Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 0, partNo: 2, Offset: 0 is stored on disk
2025-11-14T16:03:02.238643Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:930: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-11-14T16:03:02.238649Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:972: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-11-14T16:03:02.238653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.238656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.238659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.238663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.238665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.238673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.238689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.238727Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--topic1' partition 0 user user offset 0 partno 0 count 1 size 1024000 endOffset 1 max time lag 0ms effective offset 0
2025-11-14T16:03:02.238741Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 1048780 count 1 last offset 0, current partition end offset: 1
2025-11-14T16:03:02.238744Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-11-14T16:03:02.238754Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.238758Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 3. All 1 blobs are from cache.
2025-11-14T16:03:02.238773Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 1 blobs
2025-11-14T16:03:02.238898Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.238909Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.238941Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 1 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.239075Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 1 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.239091Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 1 count 1 size 24704 from pos 0 cbcount 1
2025-11-14T16:03:02.239109Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1022: Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp done, result 1763136182225 queuesize 0 startOffset 0
2025-11-14T16:03:02.239005Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 0
    Offset: 0
    PartNo: 0
    WriteTimestampMS: 1763136182225
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 10
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-14T16:03:02.239461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852025983:3356] destroyed
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.240153Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.240680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852025989:3359], now have 1 active actors on pipe
2025-11-14T16:03:02.240700Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.240707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.240741Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a3e7cb7-ed5962e5-8a84d897-7db2322_1 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.240756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.240760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.240767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.240769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.240773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.240781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.240784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.240787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.240800Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.240811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.240869Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a3e7cb7-ed5962e5-8a84d897-7db2322_1"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
2025-11-14T16:03:02.242538Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852025989:3359] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a3e7cb7-ed5962e5-8a84d897-7db2322_1"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 1
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.252544Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.256056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852025996:3362], now have 1 active actors on pipe
2025-11-14T16:03:02.256483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.256508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.256602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 1 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.257101Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 1 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.257112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 1 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.257117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 1 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.257143Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.257161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.257179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.257188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.257190Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.257195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.257198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.257203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.257234Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 0
2025-11-14T16:03:02.257245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.257249Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 1
2025-11-14T16:03:02.257252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.257255Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 2
2025-11-14T16:03:02.257495Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 1 partNo 2 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 1048790 count 1 nextOffset 2 batches 3
2025-11-14T16:03:02.257507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.257510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.257514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.257588Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 1,1 HeadOffset 1 endOffset 1 curOffset 2 d0000000000_00000000000000000001_00000_0000000001_00002? size 1048780 WTime 1763136182257
2025-11-14T16:03:02.257756Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.257769Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 1 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.261459Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 1 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.261479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.261501Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.261506Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.261515Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 1, partNo: 0, Offset: 1 is stored on disk
2025-11-14T16:03:02.261518Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.261522Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 1, partNo: 1, Offset: 1 is stored on disk
2025-11-14T16:03:02.261527Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.261530Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 1, partNo: 2, Offset: 1 is stored on disk
2025-11-14T16:03:02.261564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.261567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.261569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.261571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.261573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.261582Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.261594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.261631Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.261725Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 1
    Offset: 1
    PartNo: 0
    WriteTimestampMS: 1763136182257
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.262742Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.262761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852025996:3362] destroyed
2025-11-14T16:03:02.263073Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026001:3365], now have 1 active actors on pipe
2025-11-14T16:03:02.263205Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.263212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.263239Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|8ce025ec-3db955c-862f255b-8996dc6c_2 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.263247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.263250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.263255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.263257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.263261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.263270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.263273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.263276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.263288Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.263298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.263446Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|8ce025ec-3db955c-862f255b-8996dc6c_2"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
2025-11-14T16:03:02.263735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026001:3365] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|8ce025ec-3db955c-862f255b-8996dc6c_2"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 2
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.273566Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.274153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026004:3366], now have 1 active actors on pipe
2025-11-14T16:03:02.275140Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.275158Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.275247Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 2 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.275617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 2 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.275634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 2 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.275638Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 2 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.275660Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.275682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.275691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.275700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.275706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.275711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.275714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.275720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.275751Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 0
2025-11-14T16:03:02.275769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.275775Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 1
2025-11-14T16:03:02.275779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.275782Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 2
2025-11-14T16:03:02.275983Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 2 partNo 2 FormedBlobsCount 0 NewHead: Offset 2 PartNo 0 PackedSize 1048790 count 1 nextOffset 3 batches 3
2025-11-14T16:03:02.275999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.276001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.276005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.276055Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 2,1 HeadOffset 2 endOffset 2 curOffset 3 d0000000000_00000000000000000002_00000_0000000001_00002? size 1048780 WTime 1763136182276
2025-11-14T16:03:02.276389Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.276408Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 2 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.279590Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 2 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.279614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.279639Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.279638Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.279647Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.279658Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 2, partNo: 0, Offset: 2 is stored on disk
2025-11-14T16:03:02.279666Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.279672Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 2, partNo: 1, Offset: 2 is stored on disk
2025-11-14T16:03:02.279678Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.279695Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 2, partNo: 2, Offset: 2 is stored on disk
2025-11-14T16:03:02.279733Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.279740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.279743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.279745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.279747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.279757Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.279782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.279875Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 2
    Offset: 2
    PartNo: 0
    WriteTimestampMS: 1763136182275
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.280511Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026004:3366] destroyed
2025-11-14T16:03:02.280785Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.281270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026009:3369], now have 1 active actors on pipe
2025-11-14T16:03:02.281337Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.281348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.281382Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|6623a17e-81c32c20-4348dbea-92288204_3 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.281395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.281400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.281405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.281411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.281415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.281427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.281433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.281437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.281452Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.281469Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.281524Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.281780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026009:3369] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|6623a17e-81c32c20-4348dbea-92288204_3"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|6623a17e-81c32c20-4348dbea-92288204_3"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 3
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.289022Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.289513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026012:3370], now have 1 active actors on pipe
2025-11-14T16:03:02.291307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.291347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.291433Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 3 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.291913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 3 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.291933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 3 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.291942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 3 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.291973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.291983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.291986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.291989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.291992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.292006Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.292027Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.292034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.292043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.292050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.292053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.292055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.292059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.292093Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 0
2025-11-14T16:03:02.292111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.292118Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 1
2025-11-14T16:03:02.292122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.292126Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 2
2025-11-14T16:03:02.292376Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 3 partNo 2 FormedBlobsCount 0 NewHead: Offset 3 PartNo 0 PackedSize 1048790 count 1 nextOffset 4 batches 3
2025-11-14T16:03:02.292398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.292401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.292405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.292466Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 3,1 HeadOffset 3 endOffset 3 curOffset 4 d0000000000_00000000000000000003_00000_0000000001_00002? size 1048780 WTime 1763136182292
2025-11-14T16:03:02.292618Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.292640Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 3 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.295757Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 3 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.295775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.295796Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.295801Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.295808Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 3, partNo: 0, Offset: 3 is stored on disk
2025-11-14T16:03:02.295811Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.295815Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 3, partNo: 1, Offset: 3 is stored on disk
2025-11-14T16:03:02.295818Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.295820Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 3, partNo: 2, Offset: 3 is stored on disk
2025-11-14T16:03:02.295849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.295855Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.295857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.295858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.295859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.295869Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.295880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.295921Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.295981Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.296348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026012:3370] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 3
    Offset: 3
    PartNo: 0
    WriteTimestampMS: 1763136182291
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.296866Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.297374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026017:3373], now have 1 active actors on pipe
2025-11-14T16:03:02.297386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.297390Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.297413Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a4088e6b-2eb74f94-fdf854d1-8438e3e_4 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.297423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.297425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.297429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.297431Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.297433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.297439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.297445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.297447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.297456Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.297465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.297538Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a4088e6b-2eb74f94-fdf854d1-8438e3e_4"
    Status: Active
    SeqNo: 0
  }
}
2025-11-14T16:03:02.297810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026017:3373] destroyed
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a4088e6b-2eb74f94-fdf854d1-8438e3e_4"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 4
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.303327Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.303787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026020:3374], now have 1 active actors on pipe
2025-11-14T16:03:02.304745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.304762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.304837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 4 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.305622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 4 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.305639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 4 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.305643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 4 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.305668Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.305688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.305691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.305699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.305702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.305707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.305709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.305715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.305743Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 0
2025-11-14T16:03:02.305755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.305758Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 1
2025-11-14T16:03:02.305761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.305764Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 2
2025-11-14T16:03:02.305990Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 4 partNo 2 FormedBlobsCount 0 NewHead: Offset 4 PartNo 0 PackedSize 1048790 count 1 nextOffset 5 batches 3
2025-11-14T16:03:02.306000Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.306006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.306010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.306082Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 4,1 HeadOffset 4 endOffset 4 curOffset 5 d0000000000_00000000000000000004_00000_0000000001_00002? size 1048780 WTime 1763136182305
2025-11-14T16:03:02.306479Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.306492Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 4 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.311550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 4 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.311586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.311592Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.311611Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.311618Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.311810Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.311628Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 4, partNo: 0, Offset: 4 is stored on disk
2025-11-14T16:03:02.311631Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.311636Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 4, partNo: 1, Offset: 4 is stored on disk
2025-11-14T16:03:02.311642Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.311645Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 4, partNo: 2, Offset: 4 is stored on disk
2025-11-14T16:03:02.311682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.311685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.311688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.311690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.311694Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.311703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.311718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 4
    Offset: 4
    PartNo: 0
    WriteTimestampMS: 1763136182305
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.312746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026020:3374] destroyed
2025-11-14T16:03:02.313381Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.313824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026026:3378], now have 1 active actors on pipe
2025-11-14T16:03:02.313944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.313953Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.313987Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9ad3d202-40b6e8c9-32f6f4ea-4da59669_5 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.313998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.314001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.314007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.314009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.314013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.314023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.314026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.314028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.314042Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.314056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.314118Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9ad3d202-40b6e8c9-32f6f4ea-4da59669_5"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9ad3d202-40b6e8c9-32f6f4ea-4da59669_5"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 5
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.320139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026026:3378] destroyed
2025-11-14T16:03:02.320618Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.321148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026029:3379], now have 1 active actors on pipe
2025-11-14T16:03:02.322394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.322409Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.322481Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 5 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.322555Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 5 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.322578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 5 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.322582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 5 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.322603Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.322621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.322624Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.322634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.322637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.322641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.322644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.322649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.322677Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 0
2025-11-14T16:03:02.322689Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.322692Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 1
2025-11-14T16:03:02.322696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.322699Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 2
2025-11-14T16:03:02.322953Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 5 partNo 2 FormedBlobsCount 0 NewHead: Offset 5 PartNo 0 PackedSize 1048790 count 1 nextOffset 6 batches 3
2025-11-14T16:03:02.322966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.322968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.322972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.323037Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 5,1 HeadOffset 5 endOffset 5 curOffset 6 d0000000000_00000000000000000005_00000_0000000001_00002? size 1048780 WTime 1763136182322
2025-11-14T16:03:02.323233Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.323244Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 5 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.326932Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 5 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.326955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.326980Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.326987Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.326997Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 5, partNo: 0, Offset: 5 is stored on disk
2025-11-14T16:03:02.327001Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.327006Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 5, partNo: 1, Offset: 5 is stored on disk
2025-11-14T16:03:02.327012Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.327017Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 5, partNo: 2, Offset: 5 is stored on disk
2025-11-14T16:03:02.327051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.327054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.327057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.327060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.327062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.327072Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.327087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.327213Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.327298Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.327726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026029:3379] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 5
    Offset: 5
    PartNo: 0
    WriteTimestampMS: 1763136182322
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.329661Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.330139Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026034:3382], now have 1 active actors on pipe
2025-11-14T16:03:02.330314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.330323Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.330353Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|aa9ccae-8896d54c-76d1dd0d-9b1b9a2f_6 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.330362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.330365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.330370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.330372Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.330376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.330385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.330388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.330391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.330404Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.330422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
2025-11-14T16:03:02.330560Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|aa9ccae-8896d54c-76d1dd0d-9b1b9a2f_6"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
2025-11-14T16:03:02.330921Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026034:3382] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|aa9ccae-8896d54c-76d1dd0d-9b1b9a2f_6"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 6
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.342298Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.344964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026040:3383], now have 1 active actors on pipe
2025-11-14T16:03:02.345351Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.345363Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.345441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 6 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.345502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 6 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.345513Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 6 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.345518Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 6 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.345538Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.345558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.345563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.345572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.345576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.345580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.345583Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.345587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.345619Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 0
2025-11-14T16:03:02.345631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.345635Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 1
2025-11-14T16:03:02.345638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.345644Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 2
2025-11-14T16:03:02.346372Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 6 partNo 2 FormedBlobsCount 0 NewHead: Offset 6 PartNo 0 PackedSize 1048790 count 1 nextOffset 7 batches 3
2025-11-14T16:03:02.346385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.346388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.346391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.346462Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 6,1 HeadOffset 6 endOffset 6 curOffset 7 d0000000000_00000000000000000006_00000_0000000001_00002? size 1048780 WTime 1763136182346
2025-11-14T16:03:02.346636Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.346648Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 6 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.349775Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 6 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.349804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.349827Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.349834Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.349844Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 6, partNo: 0, Offset: 6 is stored on disk
2025-11-14T16:03:02.349849Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.349855Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 6, partNo: 1, Offset: 6 is stored on disk
2025-11-14T16:03:02.349860Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.349865Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 6, partNo: 2, Offset: 6 is stored on disk
2025-11-14T16:03:02.349897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.349901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.349905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.349907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.349910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.349919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.349935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.350067Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.350139Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 6
    Offset: 6
    PartNo: 0
    WriteTimestampMS: 1763136182345
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.351164Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026040:3383] destroyed
2025-11-14T16:03:02.351314Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.351637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026045:3386], now have 1 active actors on pipe
2025-11-14T16:03:02.351859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.351868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.351902Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|2026516f-2c36aa93-7cdfd622-84f2bb2a_7 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.351911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.351915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.351920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.351922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.351926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.351935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.351938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.351942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.351955Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.351967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.352011Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.352204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026045:3386] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|2026516f-2c36aa93-7cdfd622-84f2bb2a_7"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|2026516f-2c36aa93-7cdfd622-84f2bb2a_7"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 7
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.359472Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.360003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026049:3388], now have 1 active actors on pipe
2025-11-14T16:03:02.361237Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.361259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.361339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 7 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.361405Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 7 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.361415Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 7 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.361420Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 7 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.361442Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.361457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.361460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.361467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.361469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.361471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.361472Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.361476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.361501Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 0
2025-11-14T16:03:02.361512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.361516Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 1
2025-11-14T16:03:02.361519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.361521Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 2
2025-11-14T16:03:02.362165Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 7 partNo 2 FormedBlobsCount 0 NewHead: Offset 7 PartNo 0 PackedSize 1048790 count 1 nextOffset 8 batches 3
2025-11-14T16:03:02.362176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.362178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.362180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.362246Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 7,1 HeadOffset 7 endOffset 7 curOffset 8 d0000000000_00000000000000000007_00000_0000000001_00002? size 1048780 WTime 1763136182362
2025-11-14T16:03:02.362423Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.362434Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 7 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.365464Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 7 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.365484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.365502Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.365506Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.365514Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.365522Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 7, partNo: 0, Offset: 7 is stored on disk
2025-11-14T16:03:02.365535Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.365545Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 7, partNo: 1, Offset: 7 is stored on disk
2025-11-14T16:03:02.365551Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.365556Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 7, partNo: 2, Offset: 7 is stored on disk
2025-11-14T16:03:02.365588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.365597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.365598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.365601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.365603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.365613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000000_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.365616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000001_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.365619Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000002_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.365622Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000003_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.365625Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000004_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.365627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000005_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.365630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000006_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.365632Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000007_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.365634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 8 keys were taken away. Let's read 8390240 bytes
2025-11-14T16:03:02.365650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.365670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 8 blobs
2025-11-14T16:03:02.365684Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000000_00000_0000000001_00002?
2025-11-14T16:03:02.365697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000001_00000_0000000001_00002?
2025-11-14T16:03:02.365713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000002_00000_0000000001_00002?
2025-11-14T16:03:02.365731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000003_00000_0000000001_00002?
2025-11-14T16:03:02.365742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000004_00000_0000000001_00002?
2025-11-14T16:03:02.365782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000005_00000_0000000001_00002?
2025-11-14T16:03:02.365794Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000006_00000_0000000001_00002?
2025-11-14T16:03:02.365803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000007_00000_0000000001_00002?
2025-11-14T16:03:02.365813Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 8 blobs for compaction
2025-11-14T16:03:02.365825Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 1 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.365830Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 1 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.365835Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 2 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.365851Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 3 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.365859Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 4 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.365862Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 5 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.365866Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 6 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.365869Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 7 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.365881Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 8 blobs are from cache.
2025-11-14T16:03:02.365908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-14T16:03:02.365927Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.365931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/8] d0000000000_00000000000000000000_00000_0000000001_00002?
2025-11-14T16:03:02.365934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.365937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000000_00000_0000000001_00002?, parameters.CurOffset=0
2025-11-14T16:03:02.365938Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.365942Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.365945Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.365948Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.365951Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.365955Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.365958Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.366036Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 7
    Offset: 7
    PartNo: 0
    WriteTimestampMS: 1763136182361
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.368652Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.366076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 0.0/3
2025-11-14T16:03:02.366090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 0
2025-11-14T16:03:02.366097Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 0:0 appended
2025-11-14T16:03:02.366240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 0.1/3
2025-11-14T16:03:02.366244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 1
2025-11-14T16:03:02.366247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 0:1 appended
2025-11-14T16:03:02.366265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 0.2/3
2025-11-14T16:03:02.366267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 2
2025-11-14T16:03:02.366481Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 0 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1048790 count 1 nextOffset 1 batches 3
2025-11-14T16:03:02.366488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 0:2 appended
2025-11-14T16:03:02.366491Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/8] d0000000000_00000000000000000001_00000_0000000001_00002?
2025-11-14T16:03:02.366495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.366499Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000001_00000_0000000001_00002?, parameters.CurOffset=1
2025-11-14T16:03:02.366627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 1.0/3
2025-11-14T16:03:02.366633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 0
2025-11-14T16:03:02.366637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 1:0 appended
2025-11-14T16:03:02.366785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 1.1/3
2025-11-14T16:03:02.366790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 1
2025-11-14T16:03:02.366792Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 1:1 appended
2025-11-14T16:03:02.366811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 1.2/3
2025-11-14T16:03:02.366814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 2
2025-11-14T16:03:02.367002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 1 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2097545 count 2 nextOffset 2 batches 5
2025-11-14T16:03:02.367008Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 1:2 appended
2025-11-14T16:03:02.367013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/8] d0000000000_00000000000000000002_00000_0000000001_00002?
2025-11-14T16:03:02.367015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.367017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000002_00000_0000000001_00002?, parameters.CurOffset=2
2025-11-14T16:03:02.367147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 2.0/3
2025-11-14T16:03:02.367152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 0
2025-11-14T16:03:02.367156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 2:0 appended
2025-11-14T16:03:02.367804Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 2.1/3
2025-11-14T16:03:02.367810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 1
2025-11-14T16:03:02.367816Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 2:1 appended
2025-11-14T16:03:02.367831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 2.2/3
2025-11-14T16:03:02.367833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 2
2025-11-14T16:03:02.368038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 2 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3146300 count 3 nextOffset 3 batches 7
2025-11-14T16:03:02.368044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 2:2 appended
2025-11-14T16:03:02.368048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/8] d0000000000_00000000000000000003_00000_0000000001_00002?
2025-11-14T16:03:02.368050Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.368053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000003_00000_0000000001_00002?, parameters.CurOffset=3
2025-11-14T16:03:02.368185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 3.0/3
2025-11-14T16:03:02.368190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 0
2025-11-14T16:03:02.368193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 3:0 appended
2025-11-14T16:03:02.368810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 3.1/3
2025-11-14T16:03:02.368817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 1
2025-11-14T16:03:02.368820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 3:1 appended
2025-11-14T16:03:02.368836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 3.2/3
2025-11-14T16:03:02.368840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 2
2025-11-14T16:03:02.369745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 3 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4195055 count 4 nextOffset 4 batches 9
2025-11-14T16:03:02.369764Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 3:2 appended
2025-11-14T16:03:02.369769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/8] d0000000000_00000000000000000004_00000_0000000001_00002?
2025-11-14T16:03:02.369785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.369789Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000004_00000_0000000001_00002?, parameters.CurOffset=4
2025-11-14T16:03:02.369922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 4.0/3
2025-11-14T16:03:02.369927Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 0
2025-11-14T16:03:02.369930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 4:0 appended
2025-11-14T16:03:02.370172Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 4.1/3
2025-11-14T16:03:02.370181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 1
2025-11-14T16:03:02.370184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 4:1 appended
2025-11-14T16:03:02.370198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 4.2/3
2025-11-14T16:03:02.370201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 2
2025-11-14T16:03:02.370580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 4 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 5243810 count 5 nextOffset 5 batches 11
2025-11-14T16:03:02.370586Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 4:2 appended
2025-11-14T16:03:02.370590Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/8] d0000000000_00000000000000000005_00000_0000000001_00002?
2025-11-14T16:03:02.370592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.370597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000005_00000_0000000001_00002?, parameters.CurOffset=5
2025-11-14T16:03:02.370738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 5.0/3
2025-11-14T16:03:02.370743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 0
2025-11-14T16:03:02.370745Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 5:0 appended
2025-11-14T16:03:02.371116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 5.1/3
2025-11-14T16:03:02.371122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 1
2025-11-14T16:03:02.371125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 5:1 appended
2025-11-14T16:03:02.371138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 5.2/3
2025-11-14T16:03:02.371141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 2
2025-11-14T16:03:02.372130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 5 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 6292565 count 6 nextOffset 6 batches 13
2025-11-14T16:03:02.372138Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 5:2 appended
2025-11-14T16:03:02.372142Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/8] d0000000000_00000000000000000006_00000_0000000001_00002?
2025-11-14T16:03:02.372144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.372148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000006_00000_0000000001_00002?, parameters.CurOffset=6
2025-11-14T16:03:02.372270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 6.0/3
2025-11-14T16:03:02.372274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 0
2025-11-14T16:03:02.372278Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 6:0 appended
2025-11-14T16:03:02.372837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 6.1/3
2025-11-14T16:03:02.372843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 1
2025-11-14T16:03:02.372846Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 6:1 appended
2025-11-14T16:03:02.372862Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 6.2/3
2025-11-14T16:03:02.372865Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 2
2025-11-14T16:03:02.373647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 6 partNo 2 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 7341320 count 7 nextOffset 7 batches 15
2025-11-14T16:03:02.373662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 6:2 appended
2025-11-14T16:03:02.373667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/8] d0000000000_00000000000000000007_00000_0000000001_00002?
2025-11-14T16:03:02.373669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.373672Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000007_00000_0000000001_00002?, parameters.CurOffset=7
2025-11-14T16:03:02.373807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 7.0/3
2025-11-14T16:03:02.373814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 0
2025-11-14T16:03:02.373817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 7:0 appended
2025-11-14T16:03:02.374344Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 7.1/3
2025-11-14T16:03:02.374348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 1
2025-11-14T16:03:02.374349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 7:1 appended
2025-11-14T16:03:02.374361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 7.2/3
2025-11-14T16:03:02.374364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 2
2025-11-14T16:03:02.383830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 7 partNo 2 result is x0000000000_00000000000000000000_00000_0000000007_00016 size 8365339
2025-11-14T16:03:02.383854Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1104: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000000_00000_0000000007_00016 new key d0000000000_00000000000000000000_00000_0000000007_00016 size 8365339 WTime 1763136182383
2025-11-14T16:03:02.383895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 7 partNo 2 FormedBlobsCount 1 NewHead: Offset 7 PartNo 2 PackedSize 24736 count 1 nextOffset 8 batches 1
2025-11-14T16:03:02.383901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 7:2 appended
2025-11-14T16:03:02.383948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 7,1 HeadOffset 0 endOffset 0 curOffset 8 d0000000000_00000000000000000007_00002_0000000001_00000| size 24726 WTime 1763136182383
2025-11-14T16:03:02.383996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026049:3388] destroyed
2025-11-14T16:03:02.384010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026055:3392], now have 1 active actors on pipe
2025-11-14T16:03:02.384029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.384039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.384051Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.384065Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 7 size 8365339
2025-11-14T16:03:02.384077Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 7 partNo 2 count 1 size 24726
2025-11-14T16:03:02.384108Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|39b4c447-d3ef429e-ff4f794f-8511f580_8 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.384123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.384126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.384132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.384134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.384137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.384145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.384147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.384149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.384160Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.384200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|39b4c447-d3ef429e-ff4f794f-8511f580_8"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
2025-11-14T16:03:02.385641Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026055:3392] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|39b4c447-d3ef429e-ff4f794f-8511f580_8"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 8
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.385243Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.392262Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.392279Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.392282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.392286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.392289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.397761Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.401400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026060:3394], now have 1 active actors on pipe
2025-11-14T16:03:02.402846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.402868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.402944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 8 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.403019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 8 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.403038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 8 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.403042Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 8 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.403076Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.403093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.403097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.403104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.403106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.403110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.403112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.403116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.403144Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 0
2025-11-14T16:03:02.403155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.403159Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 1
2025-11-14T16:03:02.403162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.403164Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 2
2025-11-14T16:03:02.403397Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 8 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 1048790 count 1 nextOffset 9 batches 3
2025-11-14T16:03:02.403408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.403411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.403414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.403475Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 8,1 HeadOffset 8 endOffset 8 curOffset 9 d0000000000_00000000000000000008_00000_0000000001_00002? size 1048780 WTime 1763136182403
2025-11-14T16:03:02.403795Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.403809Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 8 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.405610Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 0 count 7 size 8365339 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.405631Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 7 count 1 size 24726 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.405652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.405705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2289: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-14T16:03:02.405709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.405814Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 7 parts 16 suffix '0' size 8365339
2025-11-14T16:03:02.405821Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 7 partno 2 count 1 parts 0 suffix '124' size 24726
2025-11-14T16:03:02.408372Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.407942Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 8 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.407973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.407994Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.408006Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.408014Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.408026Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 8, partNo: 0, Offset: 8 is stored on disk
2025-11-14T16:03:02.408030Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.408034Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 8, partNo: 1, Offset: 8 is stored on disk
2025-11-14T16:03:02.408040Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.408044Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 8, partNo: 2, Offset: 8 is stored on disk
2025-11-14T16:03:02.408090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.408094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.408099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.408102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.408104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.408133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.408165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.408181Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.408189Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000000_00000_0000000001_00002?(+) to d0000000000_00000000000000000000_00000_0000000001_00002?(+)
2025-11-14T16:03:02.408191Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000001_00000_0000000001_00002?(+) to d0000000000_00000000000000000001_00000_0000000001_00002?(+)
2025-11-14T16:03:02.408194Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000002_00000_0000000001_00002?(+) to d0000000000_00000000000000000002_00000_0000000001_00002?(+)
2025-11-14T16:03:02.408196Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000003_00000_0000000001_00002?(+) to d0000000000_00000000000000000003_00000_0000000001_00002?(+)
2025-11-14T16:03:02.408198Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000004_00000_0000000001_00002?(+) to d0000000000_00000000000000000004_00000_0000000001_00002?(+)
2025-11-14T16:03:02.408201Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000005_00000_0000000001_00002?(+) to d0000000000_00000000000000000005_00000_0000000001_00002?(+)
2025-11-14T16:03:02.408203Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000006_00000_0000000001_00002?(+) to d0000000000_00000000000000000006_00000_0000000001_00002?(+)
2025-11-14T16:03:02.408206Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000007_00000_0000000001_00002?(+) to d0000000000_00000000000000000007_00000_0000000001_00002?(+)
2025-11-14T16:03:02.408746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026060:3394] destroyed
2025-11-14T16:03:02.409351Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 0 count 1 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.409371Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 1 count 1 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.409376Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 2 count 1 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.409381Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 3 count 1 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.409386Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 4 count 1 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.409392Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 5 count 1 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.409397Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 6 count 1 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.409401Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 7 count 1 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.409418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.409430Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.409439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.409441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.409444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.409447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.409449Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.409458Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.409620Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.409639Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.409645Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.409651Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.409659Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.409666Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.409685Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.409693Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 2 suffix '63' size 1048780
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 8
    Offset: 8
    PartNo: 0
    WriteTimestampMS: 1763136182403
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.411154Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.411686Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026068:3400], now have 1 active actors on pipe
2025-11-14T16:03:02.412002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.412017Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.412057Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|708668a7-170da15-d8923a3f-a4b2ef3e_9 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.412071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.412075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.412080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.412084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.412088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.412100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.412103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.412106Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.412121Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.412144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.412249Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.412536Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026068:3400] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|708668a7-170da15-d8923a3f-a4b2ef3e_9"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|708668a7-170da15-d8923a3f-a4b2ef3e_9"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 9
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.419762Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.420413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026071:3401], now have 1 active actors on pipe
2025-11-14T16:03:02.421731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.421758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.421881Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 9 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.421958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 9 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.421968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 9 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.421973Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 9 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.421993Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.422015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.422019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.422034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.422037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.422041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.422044Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.422049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.422085Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 0
2025-11-14T16:03:02.422100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.422104Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 1
2025-11-14T16:03:02.422107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.422110Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 2
2025-11-14T16:03:02.422442Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 9 partNo 2 FormedBlobsCount 0 NewHead: Offset 9 PartNo 0 PackedSize 1048790 count 1 nextOffset 10 batches 3
2025-11-14T16:03:02.422455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.422458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.422462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.422537Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 9,1 HeadOffset 9 endOffset 9 curOffset 10 d0000000000_00000000000000000009_00000_0000000001_00002? size 1048780 WTime 1763136182422
2025-11-14T16:03:02.422711Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.422724Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 9 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.426321Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 9 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.426356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.426382Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.426382Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.426394Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.426407Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 9, partNo: 0, Offset: 9 is stored on disk
2025-11-14T16:03:02.426417Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.426423Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 9, partNo: 1, Offset: 9 is stored on disk
2025-11-14T16:03:02.426429Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.426439Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 9, partNo: 2, Offset: 9 is stored on disk
2025-11-14T16:03:02.426493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.426501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.426503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.426506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.426508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.426519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.426539Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.426628Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.427021Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026071:3401] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 9
    Offset: 9
    PartNo: 0
    WriteTimestampMS: 1763136182421
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.427741Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.428236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026076:3404], now have 1 active actors on pipe
2025-11-14T16:03:02.428340Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.428354Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.428391Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ef7e2f9c-62587c8d-2a02a2e6-b29b27cc_10 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.428405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.428408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.428416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.428418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.428422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.428432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.428438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.428441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.428456Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.428473Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.428544Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ef7e2f9c-62587c8d-2a02a2e6-b29b27cc_10"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
2025-11-14T16:03:02.429519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026076:3404] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ef7e2f9c-62587c8d-2a02a2e6-b29b27cc_10"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 10
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.435546Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.436341Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026079:3405], now have 1 active actors on pipe
2025-11-14T16:03:02.437662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.437681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.437754Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 10 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.437804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 10 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.437823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 10 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.437846Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 10 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.437873Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.437896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.437905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.437914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.437916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.437920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.437922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.437928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.437956Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 0
2025-11-14T16:03:02.437973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.437981Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 1
2025-11-14T16:03:02.437985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.437987Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 2
2025-11-14T16:03:02.438271Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 10 partNo 2 FormedBlobsCount 0 NewHead: Offset 10 PartNo 0 PackedSize 1048790 count 1 nextOffset 11 batches 3
2025-11-14T16:03:02.438290Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.438292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.438296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.438360Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 10,1 HeadOffset 10 endOffset 10 curOffset 11 d0000000000_00000000000000000010_00000_0000000001_00002? size 1048780 WTime 1763136182438
2025-11-14T16:03:02.438526Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.438547Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 10 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.443091Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 10 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.443120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.443136Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.443143Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.443162Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.443174Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 10, partNo: 0, Offset: 10 is stored on disk
2025-11-14T16:03:02.443181Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.443187Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 10, partNo: 1, Offset: 10 is stored on disk
2025-11-14T16:03:02.443191Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.443195Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 10, partNo: 2, Offset: 10 is stored on disk
2025-11-14T16:03:02.443228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.443234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.443237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.443239Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.443242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.443248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.443261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.443342Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.443632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026079:3405] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 10
    Offset: 10
    PartNo: 0
    WriteTimestampMS: 1763136182437
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.444357Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.444723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026084:3408], now have 1 active actors on pipe
2025-11-14T16:03:02.444838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.444851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.444888Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f17afa52-5017ce78-6595a1b6-a50b0fc9_11 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.444918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.444922Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.444926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.444928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.444931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.444937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.444940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.444943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.444953Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.444963Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.445020Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.445289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026084:3408] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f17afa52-5017ce78-6595a1b6-a50b0fc9_11"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f17afa52-5017ce78-6595a1b6-a50b0fc9_11"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 11
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.451712Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.452214Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026087:3409], now have 1 active actors on pipe
2025-11-14T16:03:02.453509Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.453527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.453592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 11 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.453652Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 11 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.453668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 11 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.453672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 11 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.453688Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.453712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.453719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.453725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.453728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.453732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.453734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.453739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.453760Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 0
2025-11-14T16:03:02.453776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.453783Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 1
2025-11-14T16:03:02.453786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.453789Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 2
2025-11-14T16:03:02.454022Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 11 partNo 2 FormedBlobsCount 0 NewHead: Offset 11 PartNo 0 PackedSize 1048790 count 1 nextOffset 12 batches 3
2025-11-14T16:03:02.454041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.454043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.454045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.454101Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 11,1 HeadOffset 11 endOffset 11 curOffset 12 d0000000000_00000000000000000011_00000_0000000001_00002? size 1048780 WTime 1763136182453
2025-11-14T16:03:02.454250Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.454261Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 11 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.458959Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 11 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.458990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.459004Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.459014Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.459022Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.459031Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 11, partNo: 0, Offset: 11 is stored on disk
2025-11-14T16:03:02.459035Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.459040Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 11, partNo: 1, Offset: 11 is stored on disk
2025-11-14T16:03:02.459045Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.459050Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 11, partNo: 2, Offset: 11 is stored on disk
2025-11-14T16:03:02.459086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.459089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.459091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.459093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.459095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.459102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.459116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.459188Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 11
    Offset: 11
    PartNo: 0
    WriteTimestampMS: 1763136182453
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-14T16:03:02.459760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026087:3409] destroyed
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.460490Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.461270Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026092:3412], now have 1 active actors on pipe
2025-11-14T16:03:02.461388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.461400Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.461431Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|7d29da5e-b8bc6a26-d65e0841-d1cc42d8_12 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.461442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.461445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.461451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.461454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.461457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.461467Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.461470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.461473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.461486Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.461498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.461644Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|7d29da5e-b8bc6a26-d65e0841-d1cc42d8_12"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
2025-11-14T16:03:02.462285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026092:3412] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|7d29da5e-b8bc6a26-d65e0841-d1cc42d8_12"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 12
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.468931Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.469472Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026095:3413], now have 1 active actors on pipe
2025-11-14T16:03:02.470722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.470741Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.470822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 12 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.470886Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 12 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.470906Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 12 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.470915Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 12 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.470940Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.470979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.470988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.470999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.471001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.471005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.471008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.471012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.471053Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 0
2025-11-14T16:03:02.471071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.471075Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 1
2025-11-14T16:03:02.471078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.471080Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 2
2025-11-14T16:03:02.471292Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 12 partNo 2 FormedBlobsCount 0 NewHead: Offset 12 PartNo 0 PackedSize 1048790 count 1 nextOffset 13 batches 3
2025-11-14T16:03:02.471311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.471314Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.471318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.471387Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 12,1 HeadOffset 12 endOffset 12 curOffset 13 d0000000000_00000000000000000012_00000_0000000001_00002? size 1048780 WTime 1763136182471
2025-11-14T16:03:02.471587Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.471606Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 12 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.475440Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 12 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.475470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.475488Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.475495Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.475502Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.475512Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 12, partNo: 0, Offset: 12 is stored on disk
2025-11-14T16:03:02.475516Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.475521Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 12, partNo: 1, Offset: 12 is stored on disk
2025-11-14T16:03:02.475527Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.475532Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 12, partNo: 2, Offset: 12 is stored on disk
2025-11-14T16:03:02.475565Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.475568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.475571Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.475573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.475575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.475585Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.475600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.475677Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.476077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026095:3413] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 12
    Offset: 12
    PartNo: 0
    WriteTimestampMS: 1763136182470
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.477061Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.477520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026101:3417], now have 1 active actors on pipe
2025-11-14T16:03:02.477617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.477628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.477661Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9851540e-b5de6291-558920a-b00d135d_13 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.477669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.477673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.477678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.477680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.477683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.477692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.477704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.477707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.477720Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.477731Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.477988Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.478210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026101:3417] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9851540e-b5de6291-558920a-b00d135d_13"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9851540e-b5de6291-558920a-b00d135d_13"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 13
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.485101Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.485832Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026104:3418], now have 1 active actors on pipe
2025-11-14T16:03:02.487195Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.487217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.487295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 13 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.487359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 13 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.487372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 13 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.487377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 13 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.487401Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.487418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.487422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.487430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.487433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.487437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.487440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.487445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.487471Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 0
2025-11-14T16:03:02.487484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.487488Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 1
2025-11-14T16:03:02.487491Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.487495Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 2
2025-11-14T16:03:02.487718Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 13 partNo 2 FormedBlobsCount 0 NewHead: Offset 13 PartNo 0 PackedSize 1048790 count 1 nextOffset 14 batches 3
2025-11-14T16:03:02.487728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.487731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.487734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.487806Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 13,1 HeadOffset 13 endOffset 13 curOffset 14 d0000000000_00000000000000000013_00000_0000000001_00002? size 1048780 WTime 1763136182486
2025-11-14T16:03:02.487977Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.487989Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 13 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.492682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2289: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-14T16:03:02.492701Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 13 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.492714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.492732Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.492738Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.492742Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.492746Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 13, partNo: 0, Offset: 13 is stored on disk
2025-11-14T16:03:02.492749Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.492754Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 13, partNo: 1, Offset: 13 is stored on disk
2025-11-14T16:03:02.492758Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.492762Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 13, partNo: 2, Offset: 13 is stored on disk
2025-11-14T16:03:02.492791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.492793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.492795Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.492798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.492800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.492807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.492818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.493013Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.493266Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026104:3418] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 13
    Offset: 13
    PartNo: 0
    WriteTimestampMS: 1763136182486
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.493910Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.494592Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026109:3421], now have 1 active actors on pipe
2025-11-14T16:03:02.494611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.494617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.494647Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|2f3e216c-fab4e476-79064d30-1bf2086d_14 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.494656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.494660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.494665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.494667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.494672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.494680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.494684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.494687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.494698Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.494711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.494915Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|2f3e216c-fab4e476-79064d30-1bf2086d_14"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
2025-11-14T16:03:02.495826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026109:3421] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|2f3e216c-fab4e476-79064d30-1bf2086d_14"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 14
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.502760Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.504312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026112:3422], now have 1 active actors on pipe
2025-11-14T16:03:02.504660Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.504668Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.504745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 14 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.504806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 14 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.504823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 14 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.504827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 14 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.504847Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.504863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.504867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.504874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.504877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.504881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.504883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.504888Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.504934Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 0
2025-11-14T16:03:02.504949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.504952Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 1
2025-11-14T16:03:02.504955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.504957Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 2
2025-11-14T16:03:02.505196Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 14 partNo 2 FormedBlobsCount 0 NewHead: Offset 14 PartNo 0 PackedSize 1048790 count 1 nextOffset 15 batches 3
2025-11-14T16:03:02.505207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.505210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.505214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.505289Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 14,1 HeadOffset 14 endOffset 14 curOffset 15 d0000000000_00000000000000000014_00000_0000000001_00002? size 1048780 WTime 1763136182504
2025-11-14T16:03:02.505433Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.505445Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 14 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.509116Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 14 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.509145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.509180Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.509182Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.509187Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.509196Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 14, partNo: 0, Offset: 14 is stored on disk
2025-11-14T16:03:02.509199Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.509203Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 14, partNo: 1, Offset: 14 is stored on disk
2025-11-14T16:03:02.509208Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.509211Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 14, partNo: 2, Offset: 14 is stored on disk
2025-11-14T16:03:02.509241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.509243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.509245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.509247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.509249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.509256Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.509269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.509450Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 14
    Offset: 14
    PartNo: 0
    WriteTimestampMS: 1763136182504
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.510003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026112:3422] destroyed
2025-11-14T16:03:02.510314Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.510766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026117:3425], now have 1 active actors on pipe
2025-11-14T16:03:02.510842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.510850Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.510879Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|7b7fa6e2-2e03781-8388efe6-28c0e0ee_15 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.510887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.510890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.510894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.510897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.510900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.510908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.510911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.510914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.510926Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.510936Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.510975Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|7b7fa6e2-2e03781-8388efe6-28c0e0ee_15"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
2025-11-14T16:03:02.511177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026117:3425] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|7b7fa6e2-2e03781-8388efe6-28c0e0ee_15"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 15
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.518259Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.518775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026120:3426], now have 1 active actors on pipe
2025-11-14T16:03:02.519835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.519861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.519939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 15 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.520007Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 15 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.520022Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 15 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.520028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 15 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.520049Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.520065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.520069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.520078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.520081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.520085Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.520087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.520092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.520123Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 0
2025-11-14T16:03:02.520136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.520139Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 1
2025-11-14T16:03:02.520142Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.520146Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 2
2025-11-14T16:03:02.520401Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 15 partNo 2 FormedBlobsCount 0 NewHead: Offset 15 PartNo 0 PackedSize 1048790 count 1 nextOffset 16 batches 3
2025-11-14T16:03:02.520412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.520415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.520419Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.520504Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 15,1 HeadOffset 15 endOffset 15 curOffset 16 d0000000000_00000000000000000015_00000_0000000001_00002? size 1048780 WTime 1763136182520
2025-11-14T16:03:02.520681Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.520694Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 15 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.524879Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 15 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.524921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.524934Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.524943Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.524950Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.524960Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 15, partNo: 0, Offset: 15 is stored on disk
2025-11-14T16:03:02.524964Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.524972Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 15, partNo: 1, Offset: 15 is stored on disk
2025-11-14T16:03:02.524978Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.524982Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 15, partNo: 2, Offset: 15 is stored on disk
2025-11-14T16:03:02.525015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.525018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.525020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.525023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.525025Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.525034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000008_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.525038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000009_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.525041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000010_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.525046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000011_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.525049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000012_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.525052Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000013_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.525055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000014_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.525057Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000015_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.525061Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 8 keys were taken away. Let's read 8390240 bytes
2025-11-14T16:03:02.525076Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.525100Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 8 blobs
2025-11-14T16:03:02.525103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000008_00000_0000000001_00002?
2025-11-14T16:03:02.525155Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.525114Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000009_00000_0000000001_00002?
2025-11-14T16:03:02.525132Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000010_00000_0000000001_00002?
2025-11-14T16:03:02.525145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000011_00000_0000000001_00002?
2025-11-14T16:03:02.525154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000012_00000_0000000001_00002?
2025-11-14T16:03:02.525185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000013_00000_0000000001_00002?
2025-11-14T16:03:02.525194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000014_00000_0000000001_00002?
2025-11-14T16:03:02.525203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000015_00000_0000000001_00002?
2025-11-14T16:03:02.525213Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 8 blobs for compaction
2025-11-14T16:03:02.525223Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 8 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.525227Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 9 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.525230Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 10 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.525232Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 11 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.525238Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 12 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.525242Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 13 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.525244Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 14 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.525247Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 15 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.525261Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 8 blobs are from cache.
2025-11-14T16:03:02.525268Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.525271Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.525275Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.525278Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.525281Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.525284Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.525287Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.525287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-14T16:03:02.525290Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.525295Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/8] d0000000000_00000000000000000008_00000_0000000001_00002?
2025-11-14T16:03:02.525297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.525300Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000008_00000_0000000001_00002?, parameters.CurOffset=8
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 15
    Offset: 15
    PartNo: 0
    WriteTimestampMS: 1763136182519
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.525984Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.525453Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 8.0/3
2025-11-14T16:03:02.525470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 0
2025-11-14T16:03:02.525476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 8:0 appended
2025-11-14T16:03:02.525662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 8.1/3
2025-11-14T16:03:02.525667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 1
2025-11-14T16:03:02.525671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 8:1 appended
2025-11-14T16:03:02.525689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 8.2/3
2025-11-14T16:03:02.525693Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 2
2025-11-14T16:03:02.525917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 8 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 1048790 count 1 nextOffset 9 batches 3
2025-11-14T16:03:02.525922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 8:2 appended
2025-11-14T16:03:02.525925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/8] d0000000000_00000000000000000009_00000_0000000001_00002?
2025-11-14T16:03:02.525926Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.525928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000009_00000_0000000001_00002?, parameters.CurOffset=9
2025-11-14T16:03:02.526059Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 9.0/3
2025-11-14T16:03:02.526062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 0
2025-11-14T16:03:02.526063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 9:0 appended
2025-11-14T16:03:02.526196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 9.1/3
2025-11-14T16:03:02.526199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 1
2025-11-14T16:03:02.526200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 9:1 appended
2025-11-14T16:03:02.526255Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 9.2/3
2025-11-14T16:03:02.526257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 2
2025-11-14T16:03:02.526635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 9 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 2097545 count 2 nextOffset 10 batches 5
2025-11-14T16:03:02.526642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 9:2 appended
2025-11-14T16:03:02.526645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/8] d0000000000_00000000000000000010_00000_0000000001_00002?
2025-11-14T16:03:02.526646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.526650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000010_00000_0000000001_00002?, parameters.CurOffset=10
2025-11-14T16:03:02.526782Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 10.0/3
2025-11-14T16:03:02.526787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 0
2025-11-14T16:03:02.526790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 10:0 appended
2025-11-14T16:03:02.526933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 10.1/3
2025-11-14T16:03:02.526936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 1
2025-11-14T16:03:02.526938Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 10:1 appended
2025-11-14T16:03:02.526950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 10.2/3
2025-11-14T16:03:02.526953Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 2
2025-11-14T16:03:02.527122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 10 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 3146300 count 3 nextOffset 11 batches 7
2025-11-14T16:03:02.527126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 10:2 appended
2025-11-14T16:03:02.527130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/8] d0000000000_00000000000000000011_00000_0000000001_00002?
2025-11-14T16:03:02.527133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.527136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000011_00000_0000000001_00002?, parameters.CurOffset=11
2025-11-14T16:03:02.527271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 11.0/3
2025-11-14T16:03:02.527275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 0
2025-11-14T16:03:02.527277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 11:0 appended
2025-11-14T16:03:02.527409Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 11.1/3
2025-11-14T16:03:02.527412Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 1
2025-11-14T16:03:02.527414Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 11:1 appended
2025-11-14T16:03:02.527426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 11.2/3
2025-11-14T16:03:02.527428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 2
2025-11-14T16:03:02.527595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 11 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 4195055 count 4 nextOffset 12 batches 9
2025-11-14T16:03:02.527598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 11:2 appended
2025-11-14T16:03:02.527600Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/8] d0000000000_00000000000000000012_00000_0000000001_00002?
2025-11-14T16:03:02.527602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.527603Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000012_00000_0000000001_00002?, parameters.CurOffset=12
2025-11-14T16:03:02.527734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 12.0/3
2025-11-14T16:03:02.527740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 0
2025-11-14T16:03:02.527743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 12:0 appended
2025-11-14T16:03:02.527882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 12.1/3
2025-11-14T16:03:02.527885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 1
2025-11-14T16:03:02.527886Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 12:1 appended
2025-11-14T16:03:02.527896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 12.2/3
2025-11-14T16:03:02.527898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 2
2025-11-14T16:03:02.528051Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 12 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 5243810 count 5 nextOffset 13 batches 11
2025-11-14T16:03:02.528053Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 12:2 appended
2025-11-14T16:03:02.528055Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/8] d0000000000_00000000000000000013_00000_0000000001_00002?
2025-11-14T16:03:02.528056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.528058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000013_00000_0000000001_00002?, parameters.CurOffset=13
2025-11-14T16:03:02.528190Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 13.0/3
2025-11-14T16:03:02.528193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 0
2025-11-14T16:03:02.528194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 13:0 appended
2025-11-14T16:03:02.528669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 13.1/3
2025-11-14T16:03:02.528673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 1
2025-11-14T16:03:02.528675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 13:1 appended
2025-11-14T16:03:02.528685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 13.2/3
2025-11-14T16:03:02.528687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 2
2025-11-14T16:03:02.530148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 13 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 6292565 count 6 nextOffset 14 batches 13
2025-11-14T16:03:02.530166Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 13:2 appended
2025-11-14T16:03:02.530171Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/8] d0000000000_00000000000000000014_00000_0000000001_00002?
2025-11-14T16:03:02.530174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.530177Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000014_00000_0000000001_00002?, parameters.CurOffset=14
2025-11-14T16:03:02.530319Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 14.0/3
2025-11-14T16:03:02.530331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 0
2025-11-14T16:03:02.530334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 14:0 appended
2025-11-14T16:03:02.530478Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 14.1/3
2025-11-14T16:03:02.530489Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 1
2025-11-14T16:03:02.530492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 14:1 appended
2025-11-14T16:03:02.530503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 14.2/3
2025-11-14T16:03:02.530506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 2
2025-11-14T16:03:02.531106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 14 partNo 2 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 7341320 count 7 nextOffset 15 batches 15
2025-11-14T16:03:02.531112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 14:2 appended
2025-11-14T16:03:02.531116Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/8] d0000000000_00000000000000000015_00000_0000000001_00002?
2025-11-14T16:03:02.531118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.531120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000015_00000_0000000001_00002?, parameters.CurOffset=15
2025-11-14T16:03:02.531257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 15.0/3
2025-11-14T16:03:02.531262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 0
2025-11-14T16:03:02.531264Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 15:0 appended
2025-11-14T16:03:02.531601Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 15.1/3
2025-11-14T16:03:02.531606Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 1
2025-11-14T16:03:02.537878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 15 partNo 1 result is x0000000000_00000000000000000007_00002_0000000008_00015 size 7878063
2025-11-14T16:03:02.537896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 15:1 appended
2025-11-14T16:03:02.537919Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 15.2/3
2025-11-14T16:03:02.537925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 2
2025-11-14T16:03:02.537940Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1104: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000007_00002_0000000008_00015 new key d0000000000_00000000000000000007_00002_0000000008_00015 size 7878063 WTime 1763136182537
2025-11-14T16:03:02.538081Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 15 partNo 2 FormedBlobsCount 1 NewHead: Offset 15 PartNo 1 PackedSize 536763 count 1 nextOffset 16 batches 2
2025-11-14T16:03:02.538090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 15:2 appended
2025-11-14T16:03:02.538145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 15,1 HeadOffset 7 endOffset 8 curOffset 16 d0000000000_00000000000000000015_00001_0000000001_00001| size 536753 WTime 1763136182537
2025-11-14T16:03:02.538254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026120:3426] destroyed
2025-11-14T16:03:02.538263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026126:3430], now have 1 active actors on pipe
2025-11-14T16:03:02.538291Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.538298Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.538312Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.538323Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 7 partNo 2 count 8 size 7878063
2025-11-14T16:03:02.538330Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 15 partNo 1 count 1 size 536753
2025-11-14T16:03:02.538364Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f704d115-363269de-b1d621d1-418fb8fb_16 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.538371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.538374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.538379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.538381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.538385Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.538395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.538401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.538406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.538417Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.538456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.538761Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.539077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026126:3430] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f704d115-363269de-b1d621d1-418fb8fb_16"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f704d115-363269de-b1d621d1-418fb8fb_16"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 16
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.548581Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.549045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026131:3433], now have 1 active actors on pipe
2025-11-14T16:03:02.550027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.550046Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.550130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 16 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.550189Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 16 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.550198Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 16 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.550201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 16 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.550218Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.550230Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.550233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.550241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.550244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.550247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.550250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.550255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.550284Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 0
2025-11-14T16:03:02.550303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.550306Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 1
2025-11-14T16:03:02.550309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.550314Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 2
2025-11-14T16:03:02.550508Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 16 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 1048790 count 1 nextOffset 17 batches 3
2025-11-14T16:03:02.550514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.550516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.550518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.550562Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 16,1 HeadOffset 16 endOffset 16 curOffset 17 d0000000000_00000000000000000016_00000_0000000001_00002? size 1048780 WTime 1763136182547
2025-11-14T16:03:02.550715Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.550723Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 16 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.557597Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 7 count 8 size 7878063 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.557612Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 15 count 1 size 536753 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.557630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.557753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2289: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-14T16:03:02.557762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.557819Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 7 partno 2 count 8 parts 15 suffix '0' size 7878063
2025-11-14T16:03:02.557828Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 15 partno 1 count 1 parts 1 suffix '124' size 536753
2025-11-14T16:03:02.561479Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 16 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.561521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.561554Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.561564Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.561579Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 16, partNo: 0, Offset: 16 is stored on disk
2025-11-14T16:03:02.561593Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.561598Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 16, partNo: 1, Offset: 16 is stored on disk
2025-11-14T16:03:02.561605Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.561609Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 16, partNo: 2, Offset: 16 is stored on disk
2025-11-14T16:03:02.561660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.561664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.561667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.561670Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.561673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.561724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.561747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.561772Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.561782Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000008_00000_0000000001_00002?(+) to d0000000000_00000000000000000008_00000_0000000001_00002?(+)
2025-11-14T16:03:02.561786Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000009_00000_0000000001_00002?(+) to d0000000000_00000000000000000009_00000_0000000001_00002?(+)
2025-11-14T16:03:02.561789Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000010_00000_0000000001_00002?(+) to d0000000000_00000000000000000010_00000_0000000001_00002?(+)
2025-11-14T16:03:02.561792Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000011_00000_0000000001_00002?(+) to d0000000000_00000000000000000011_00000_0000000001_00002?(+)
2025-11-14T16:03:02.561797Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000012_00000_0000000001_00002?(+) to d0000000000_00000000000000000012_00000_0000000001_00002?(+)
2025-11-14T16:03:02.561800Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000013_00000_0000000001_00002?(+) to d0000000000_00000000000000000013_00000_0000000001_00002?(+)
2025-11-14T16:03:02.561803Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000014_00000_0000000001_00002?(+) to d0000000000_00000000000000000014_00000_0000000001_00002?(+)
2025-11-14T16:03:02.561806Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000015_00000_0000000001_00002?(+) to d0000000000_00000000000000000015_00000_0000000001_00002?(+)
2025-11-14T16:03:02.561809Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000007_00002_0000000001_00000|(+) to d0000000000_00000000000000000007_00002_0000000001_00000|(+)
2025-11-14T16:03:02.562070Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.562135Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.562557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026131:3433] destroyed
2025-11-14T16:03:02.562994Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 8 count 1 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.563007Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 9 count 1 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.563015Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 10 count 1 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.563021Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 11 count 1 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.563027Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 12 count 1 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.563033Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 13 count 1 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.563045Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 14 count 1 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.563050Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 15 count 1 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.563057Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 7 count 1 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.563077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.563097Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.563096Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.563106Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.563110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.563111Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.563113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.563115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.563118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.563121Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.563121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.563126Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.563130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.563131Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.563138Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.563143Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.563148Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 7 partno 2 count 1 parts 0 suffix '124' size 24726
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 16
    Offset: 16
    PartNo: 0
    WriteTimestampMS: 1763136182547
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 13
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.565113Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.565684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026141:3440], now have 1 active actors on pipe
2025-11-14T16:03:02.565703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.565710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.565752Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|6bc4301c-9964ae92-97575dd7-dd49cd7f_17 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.565766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.565773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.565778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.565781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.565784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.565793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.565796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.565798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.565820Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.565839Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.565971Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|6bc4301c-9964ae92-97575dd7-dd49cd7f_17"
    Status: Active
    SeqNo: 0
  }
}
2025-11-14T16:03:02.566238Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026141:3440] destroyed
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|6bc4301c-9964ae92-97575dd7-dd49cd7f_17"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 17
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.572408Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.572828Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026144:3441], now have 1 active actors on pipe
2025-11-14T16:03:02.574342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.574366Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.574438Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 17 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.574507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 17 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.574531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 17 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.574535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 17 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.574551Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.574572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.574580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.574589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.574596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.574600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.574602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.574608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.574637Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 0
2025-11-14T16:03:02.574657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.574665Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 1
2025-11-14T16:03:02.574668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.574672Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 2
2025-11-14T16:03:02.574894Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 17 partNo 2 FormedBlobsCount 0 NewHead: Offset 17 PartNo 0 PackedSize 1048790 count 1 nextOffset 18 batches 3
2025-11-14T16:03:02.574915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.574919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.574928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.574998Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 17,1 HeadOffset 17 endOffset 17 curOffset 18 d0000000000_00000000000000000017_00000_0000000001_00002? size 1048780 WTime 1763136182574
2025-11-14T16:03:02.575171Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.575184Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 17 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.578730Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 17 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.578757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.578779Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.578790Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.578798Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 17, partNo: 0, Offset: 17 is stored on disk
2025-11-14T16:03:02.578809Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.578814Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 17, partNo: 1, Offset: 17 is stored on disk
2025-11-14T16:03:02.578823Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.578828Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 17, partNo: 2, Offset: 17 is stored on disk
2025-11-14T16:03:02.578864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.578870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.578872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.578874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.578878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.578885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.578897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.578934Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.579019Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.579279Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026144:3441] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 17
    Offset: 17
    PartNo: 0
    WriteTimestampMS: 1763136182574
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.581222Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.581602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026149:3444], now have 1 active actors on pipe
2025-11-14T16:03:02.581685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.581697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.581731Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|76ca1a16-8579f77a-e1095a90-6950c2c1_18 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.581742Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.581744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.581748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.581751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.581754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.581776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.581782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.581785Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.581794Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.581804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.581867Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.582083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026149:3444] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|76ca1a16-8579f77a-e1095a90-6950c2c1_18"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|76ca1a16-8579f77a-e1095a90-6950c2c1_18"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 18
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.588253Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.588797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026152:3445], now have 1 active actors on pipe
2025-11-14T16:03:02.589879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.589895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.589955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 18 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.590008Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 18 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.590016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 18 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.590019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 18 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.590031Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.590041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.590046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.590051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.590053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.590054Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.590056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.590058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.590072Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 0
2025-11-14T16:03:02.590079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.590081Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 1
2025-11-14T16:03:02.590083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.590085Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 2
2025-11-14T16:03:02.590282Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 18 partNo 2 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 1048790 count 1 nextOffset 19 batches 3
2025-11-14T16:03:02.590300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.590302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.590304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.590343Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 18,1 HeadOffset 18 endOffset 18 curOffset 19 d0000000000_00000000000000000018_00000_0000000001_00002? size 1048780 WTime 1763136182589
2025-11-14T16:03:02.590520Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.590539Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 18 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.593001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2289: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-14T16:03:02.594340Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 18 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.594363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.594388Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.594408Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.594420Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 18, partNo: 0, Offset: 18 is stored on disk
2025-11-14T16:03:02.594429Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.594435Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 18, partNo: 1, Offset: 18 is stored on disk
2025-11-14T16:03:02.594444Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.594453Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 18, partNo: 2, Offset: 18 is stored on disk
2025-11-14T16:03:02.594497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.594505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.594509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.594511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.594513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.594524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.594544Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.594584Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.594670Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 18
    Offset: 18
    PartNo: 0
    WriteTimestampMS: 1763136182589
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.594997Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026152:3445] destroyed
2025-11-14T16:03:02.595527Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.596093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026157:3448], now have 1 active actors on pipe
2025-11-14T16:03:02.596345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.596356Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.596391Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|32577889-101b019d-a0f64048-2ce8fcf1_19 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.596400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.596403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.596408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.596410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.596414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.596423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.596425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.596428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.596440Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.596454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
2025-11-14T16:03:02.596530Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|32577889-101b019d-a0f64048-2ce8fcf1_19"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
2025-11-14T16:03:02.596944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026157:3448] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|32577889-101b019d-a0f64048-2ce8fcf1_19"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 19
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.604202Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.604630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026160:3449], now have 1 active actors on pipe
2025-11-14T16:03:02.605782Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.605804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.605874Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 19 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.605947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 19 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.605960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 19 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.605965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 19 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.605982Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.605996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.605999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.606006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.606008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.606015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.606017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.606022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.606059Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 0
2025-11-14T16:03:02.606070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.606074Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 1
2025-11-14T16:03:02.606079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.606082Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 2
2025-11-14T16:03:02.606335Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 19 partNo 2 FormedBlobsCount 0 NewHead: Offset 19 PartNo 0 PackedSize 1048790 count 1 nextOffset 20 batches 3
2025-11-14T16:03:02.606346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.606348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.606351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.606416Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 19,1 HeadOffset 19 endOffset 19 curOffset 20 d0000000000_00000000000000000019_00000_0000000001_00002? size 1048780 WTime 1763136182606
2025-11-14T16:03:02.606588Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.606597Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 19 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.609797Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 19 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.609824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.609840Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.609844Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.609849Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.609858Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 19, partNo: 0, Offset: 19 is stored on disk
2025-11-14T16:03:02.609862Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.609867Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 19, partNo: 1, Offset: 19 is stored on disk
2025-11-14T16:03:02.609871Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.609877Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 19, partNo: 2, Offset: 19 is stored on disk
2025-11-14T16:03:02.609905Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.609918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.609921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.609924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.609925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.609931Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.609943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.609998Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.610255Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026160:3449] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 19
    Offset: 19
    PartNo: 0
    WriteTimestampMS: 1763136182605
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.610816Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.611145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026165:3452], now have 1 active actors on pipe
2025-11-14T16:03:02.611260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.611268Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.611294Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|17668f1c-4856ba22-432c0004-245cbe8c_20 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.611305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.611308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.611311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.611313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.611316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.611322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.611329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.611332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.611342Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.611350Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.611413Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|17668f1c-4856ba22-432c0004-245cbe8c_20"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
2025-11-14T16:03:02.611611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026165:3452] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|17668f1c-4856ba22-432c0004-245cbe8c_20"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 20
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.619471Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.619939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026168:3453], now have 1 active actors on pipe
2025-11-14T16:03:02.621125Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.621144Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.621240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 20 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.621294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 20 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.621307Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 20 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.621317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 20 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.621337Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.621358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.621366Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.621375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.621377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.621381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.621383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.621388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.621419Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 0
2025-11-14T16:03:02.621436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.621440Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 1
2025-11-14T16:03:02.621443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.621446Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 2
2025-11-14T16:03:02.621684Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 20 partNo 2 FormedBlobsCount 0 NewHead: Offset 20 PartNo 0 PackedSize 1048790 count 1 nextOffset 21 batches 3
2025-11-14T16:03:02.621700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.621703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.621707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.621777Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 20,1 HeadOffset 20 endOffset 20 curOffset 21 d0000000000_00000000000000000020_00000_0000000001_00002? size 1048780 WTime 1763136182620
2025-11-14T16:03:02.621966Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.621987Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 20 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.625213Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 20 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.625248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.625267Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.625274Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.625280Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.625288Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 20, partNo: 0, Offset: 20 is stored on disk
2025-11-14T16:03:02.625291Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.625295Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 20, partNo: 1, Offset: 20 is stored on disk
2025-11-14T16:03:02.625300Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.625305Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 20, partNo: 2, Offset: 20 is stored on disk
2025-11-14T16:03:02.625338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.625345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.625348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.625351Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.625353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.625363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.625387Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
2025-11-14T16:03:02.625456Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.625691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026168:3453] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 20
    Offset: 20
    PartNo: 0
    WriteTimestampMS: 1763136182620
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.626241Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.626567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026173:3456], now have 1 active actors on pipe
2025-11-14T16:03:02.626669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.626679Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.626708Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|49694664-a71f1c75-40a4f964-42a3f828_21 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.626722Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.626725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.626730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.626732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.626736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.626744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.626747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.626751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.626762Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.626772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.626813Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.627023Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026173:3456] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|49694664-a71f1c75-40a4f964-42a3f828_21"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|49694664-a71f1c75-40a4f964-42a3f828_21"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 21
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.634149Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.634608Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026176:3457], now have 1 active actors on pipe
2025-11-14T16:03:02.635773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.635789Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.635867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 21 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.635920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 21 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.635941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 21 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.635946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 21 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.635965Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.635983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.635992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.636001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.636003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.636005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.636007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.636024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.636057Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 0
2025-11-14T16:03:02.636073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.636080Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 1
2025-11-14T16:03:02.636084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.636087Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 2
2025-11-14T16:03:02.636331Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 21 partNo 2 FormedBlobsCount 0 NewHead: Offset 21 PartNo 0 PackedSize 1048790 count 1 nextOffset 22 batches 3
2025-11-14T16:03:02.636350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.636354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.636357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.636425Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 21,1 HeadOffset 21 endOffset 21 curOffset 22 d0000000000_00000000000000000021_00000_0000000001_00002? size 1048780 WTime 1763136182636
2025-11-14T16:03:02.636963Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.636981Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 21 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.640073Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 21 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.640100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.640124Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.640135Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.640136Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.640146Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 21, partNo: 0, Offset: 21 is stored on disk
2025-11-14T16:03:02.640151Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.640157Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 21, partNo: 1, Offset: 21 is stored on disk
2025-11-14T16:03:02.640162Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.640168Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 21, partNo: 2, Offset: 21 is stored on disk
2025-11-14T16:03:02.640206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.640214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.640216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.640218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.640220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.640229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.640244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.640324Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.640596Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026176:3457] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 21
    Offset: 21
    PartNo: 0
    WriteTimestampMS: 1763136182635
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.641232Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.641659Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026182:3461], now have 1 active actors on pipe
2025-11-14T16:03:02.641722Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.641732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.641766Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b2f4c5-4dde6b04-8e790e70-c84fe248_22 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.641779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.641791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.641796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.641802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.641806Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.641815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.641818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.641821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.641834Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.641849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.641893Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b2f4c5-4dde6b04-8e790e70-c84fe248_22"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
2025-11-14T16:03:02.642086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026182:3461] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b2f4c5-4dde6b04-8e790e70-c84fe248_22"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 22
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.648033Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.648442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026185:3462], now have 1 active actors on pipe
2025-11-14T16:03:02.649620Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.649661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.649738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 22 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.649813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 22 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.649830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 22 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.649836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 22 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.649858Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.649877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.649886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.649893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.649897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.649901Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.649918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.649923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.649948Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 0
2025-11-14T16:03:02.649961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.649965Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 1
2025-11-14T16:03:02.649969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.649973Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 2
2025-11-14T16:03:02.650209Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 22 partNo 2 FormedBlobsCount 0 NewHead: Offset 22 PartNo 0 PackedSize 1048790 count 1 nextOffset 23 batches 3
2025-11-14T16:03:02.650220Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.650223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.650246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.650317Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 22,1 HeadOffset 22 endOffset 22 curOffset 23 d0000000000_00000000000000000022_00000_0000000001_00002? size 1048780 WTime 1763136182649
2025-11-14T16:03:02.650497Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.650510Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 22 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.653805Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 22 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.653833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.653858Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.653864Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.653874Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 22, partNo: 0, Offset: 22 is stored on disk
2025-11-14T16:03:02.653877Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.653881Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 22, partNo: 1, Offset: 22 is stored on disk
2025-11-14T16:03:02.653888Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.653898Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 22, partNo: 2, Offset: 22 is stored on disk
2025-11-14T16:03:02.653931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.653934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.653936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.653938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.653939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.653949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.653966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.653995Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.654057Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.654383Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026185:3462] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 22
    Offset: 22
    PartNo: 0
    WriteTimestampMS: 1763136182649
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.654999Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.655455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026190:3465], now have 1 active actors on pipe
2025-11-14T16:03:02.656280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.656297Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.656331Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|cdf64303-13a8930-f6dbb57f-77115384_23 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.656346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.656349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.656355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.656357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.656362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.656375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.656383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.656386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.656399Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.656417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.656507Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|cdf64303-13a8930-f6dbb57f-77115384_23"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
2025-11-14T16:03:02.656767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026190:3465] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|cdf64303-13a8930-f6dbb57f-77115384_23"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 23
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.663604Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.664159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026193:3466], now have 1 active actors on pipe
2025-11-14T16:03:02.665702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.665724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.665818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 23 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.665882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 23 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.665902Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 23 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.665911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 23 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.665933Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.665955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.665963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.665972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.665980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.665987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.665990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.665995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.666023Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 0
2025-11-14T16:03:02.666043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.666051Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 1
2025-11-14T16:03:02.666055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.666059Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 2
2025-11-14T16:03:02.666307Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 23 partNo 2 FormedBlobsCount 0 NewHead: Offset 23 PartNo 0 PackedSize 1048790 count 1 nextOffset 24 batches 3
2025-11-14T16:03:02.666327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.666331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.666334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.666413Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 23,1 HeadOffset 23 endOffset 23 curOffset 24 d0000000000_00000000000000000023_00000_0000000001_00002? size 1048780 WTime 1763136182666
2025-11-14T16:03:02.666604Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.666618Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 23 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.669804Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 23 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.669827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.669848Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.672730Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.669853Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.669860Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 23, partNo: 0, Offset: 23 is stored on disk
2025-11-14T16:03:02.669864Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.669871Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 23, partNo: 1, Offset: 23 is stored on disk
2025-11-14T16:03:02.669876Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.669879Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 23, partNo: 2, Offset: 23 is stored on disk
2025-11-14T16:03:02.669911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.669914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.669915Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.669918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.669920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.669930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000016_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.669933Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000017_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.669936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000018_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.669940Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000019_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.669942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.669945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.669948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.669950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000023_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.669952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 8 keys were taken away. Let's read 8390240 bytes
2025-11-14T16:03:02.669965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.669973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 8 blobs
2025-11-14T16:03:02.669976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000016_00000_0000000001_00002?
2025-11-14T16:03:02.669984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000017_00000_0000000001_00002?
2025-11-14T16:03:02.669992Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000018_00000_0000000001_00002?
2025-11-14T16:03:02.670004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000019_00000_0000000001_00002?
2025-11-14T16:03:02.670011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00000_0000000001_00002?
2025-11-14T16:03:02.670028Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00002?
2025-11-14T16:03:02.670035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000001_00002?
2025-11-14T16:03:02.670042Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000023_00000_0000000001_00002?
2025-11-14T16:03:02.670049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 8 blobs for compaction
2025-11-14T16:03:02.670057Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 16 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.670060Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 17 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
CallPersQueueGRPC response:
2025-11-14T16:03:02.670063Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 18 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.670066Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 19 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.670069Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 20 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 23
    Offset: 23
    PartNo: 0
    WriteTimestampMS: 1763136182665
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-14T16:03:02.670072Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 21 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.670075Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 22 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.670077Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 23 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.670083Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 8 blobs are from cache.
2025-11-14T16:03:02.670107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-14T16:03:02.670115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/8] d0000000000_00000000000000000016_00000_0000000001_00002?
2025-11-14T16:03:02.670117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.670119Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000016_00000_0000000001_00002?, parameters.CurOffset=16
2025-11-14T16:03:02.670260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 16.0/3
2025-11-14T16:03:02.670267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 0
2025-11-14T16:03:02.670273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 16:0 appended
CallPersQueueGRPC request to localhost:27509
2025-11-14T16:03:02.670418Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 16.1/3
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.670421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 1
2025-11-14T16:03:02.670424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 16:1 appended
2025-11-14T16:03:02.670440Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 16.2/3
2025-11-14T16:03:02.670443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 2
2025-11-14T16:03:02.670718Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 16 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 1048790 count 1 nextOffset 17 batches 3
2025-11-14T16:03:02.670724Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 16:2 appended
2025-11-14T16:03:02.670728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/8] d0000000000_00000000000000000017_00000_0000000001_00002?
2025-11-14T16:03:02.670733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.670736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000017_00000_0000000001_00002?, parameters.CurOffset=17
2025-11-14T16:03:02.670883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 17.0/3
2025-11-14T16:03:02.670889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 0
2025-11-14T16:03:02.670891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 17:0 appended
2025-11-14T16:03:02.671042Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 17.1/3
2025-11-14T16:03:02.671046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 1
2025-11-14T16:03:02.671048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 17:1 appended
2025-11-14T16:03:02.671063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 17.2/3
2025-11-14T16:03:02.671067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 2
2025-11-14T16:03:02.671255Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 17 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 2097545 count 2 nextOffset 18 batches 5
2025-11-14T16:03:02.671259Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 17:2 appended
2025-11-14T16:03:02.671263Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/8] d0000000000_00000000000000000018_00000_0000000001_00002?
2025-11-14T16:03:02.671265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.671267Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000018_00000_0000000001_00002?, parameters.CurOffset=18
2025-11-14T16:03:02.671415Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 18.0/3
2025-11-14T16:03:02.671419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 0
2025-11-14T16:03:02.671423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 18:0 appended
2025-11-14T16:03:02.671823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 18.1/3
2025-11-14T16:03:02.671827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 1
2025-11-14T16:03:02.671829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 18:1 appended
2025-11-14T16:03:02.671843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 18.2/3
2025-11-14T16:03:02.671847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 2
2025-11-14T16:03:02.672524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 18 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 3146300 count 3 nextOffset 19 batches 7
2025-11-14T16:03:02.672530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 18:2 appended
2025-11-14T16:03:02.672534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/8] d0000000000_00000000000000000019_00000_0000000001_00002?
2025-11-14T16:03:02.672537Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.672541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000019_00000_0000000001_00002?, parameters.CurOffset=19
2025-11-14T16:03:02.672606Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.672620Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.672624Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.672628Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.672632Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.672636Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.672639Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.672643Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.672646Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.672688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 19.0/3
2025-11-14T16:03:02.672692Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 0
2025-11-14T16:03:02.672695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 19:0 appended
2025-11-14T16:03:02.672840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 19.1/3
2025-11-14T16:03:02.672847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 1
2025-11-14T16:03:02.672849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 19:1 appended
2025-11-14T16:03:02.672863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 19.2/3
2025-11-14T16:03:02.672866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 2
2025-11-14T16:03:02.673981Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.673881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 19 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 4195055 count 4 nextOffset 20 batches 9
2025-11-14T16:03:02.673900Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 19:2 appended
2025-11-14T16:03:02.673905Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/8] d0000000000_00000000000000000020_00000_0000000001_00002?
2025-11-14T16:03:02.673907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.673910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000020_00000_0000000001_00002?, parameters.CurOffset=20
2025-11-14T16:03:02.674069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 20.0/3
2025-11-14T16:03:02.674073Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 0
2025-11-14T16:03:02.674075Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 20:0 appended
2025-11-14T16:03:02.674225Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 20.1/3
2025-11-14T16:03:02.674238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 1
2025-11-14T16:03:02.674240Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 20:1 appended
2025-11-14T16:03:02.674252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 20.2/3
2025-11-14T16:03:02.674255Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 2
2025-11-14T16:03:02.674445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 20 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 5243810 count 5 nextOffset 21 batches 11
2025-11-14T16:03:02.674457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 20:2 appended
2025-11-14T16:03:02.674460Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/8] d0000000000_00000000000000000021_00000_0000000001_00002?
2025-11-14T16:03:02.674462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.674464Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000021_00000_0000000001_00002?, parameters.CurOffset=21
2025-11-14T16:03:02.674610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 21.0/3
2025-11-14T16:03:02.674624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 0
2025-11-14T16:03:02.674627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 21:0 appended
2025-11-14T16:03:02.674771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 21.1/3
2025-11-14T16:03:02.674776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 1
2025-11-14T16:03:02.674778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 21:1 appended
2025-11-14T16:03:02.674790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 21.2/3
2025-11-14T16:03:02.674793Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 2
2025-11-14T16:03:02.674991Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 21 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 6292565 count 6 nextOffset 22 batches 13
2025-11-14T16:03:02.674995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 21:2 appended
2025-11-14T16:03:02.675000Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/8] d0000000000_00000000000000000022_00000_0000000001_00002?
2025-11-14T16:03:02.675002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.675005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000022_00000_0000000001_00002?, parameters.CurOffset=22
2025-11-14T16:03:02.675154Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 22.0/3
2025-11-14T16:03:02.675167Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 0
2025-11-14T16:03:02.675170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 22:0 appended
2025-11-14T16:03:02.675883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 22.1/3
2025-11-14T16:03:02.675888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 1
2025-11-14T16:03:02.675891Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 22:1 appended
2025-11-14T16:03:02.675904Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 22.2/3
2025-11-14T16:03:02.675906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 2
2025-11-14T16:03:02.676627Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 22 partNo 2 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 7341320 count 7 nextOffset 23 batches 15
2025-11-14T16:03:02.676633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 22:2 appended
2025-11-14T16:03:02.676638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/8] d0000000000_00000000000000000023_00000_0000000001_00002?
2025-11-14T16:03:02.676639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.676643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000023_00000_0000000001_00002?, parameters.CurOffset=23
2025-11-14T16:03:02.676788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 23.0/3
2025-11-14T16:03:02.676803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 0
2025-11-14T16:03:02.683130Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 23 partNo 0 result is x0000000000_00000000000000000015_00001_0000000008_00015 size 7878063
2025-11-14T16:03:02.683145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 23:0 appended
2025-11-14T16:03:02.683286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 23.1/3
2025-11-14T16:03:02.683292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 1
2025-11-14T16:03:02.683296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 23:1 appended
2025-11-14T16:03:02.683313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 23.2/3
2025-11-14T16:03:02.683316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 2
2025-11-14T16:03:02.683328Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1104: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000015_00001_0000000008_00015 new key d0000000000_00000000000000000015_00001_0000000008_00015 size 7878063 WTime 1763136182683
2025-11-14T16:03:02.683565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 23 partNo 2 FormedBlobsCount 1 NewHead: Offset 23 PartNo 0 PackedSize 1048790 count 1 nextOffset 24 batches 3
2025-11-14T16:03:02.683584Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 23:2 appended
2025-11-14T16:03:02.683638Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 23,1 HeadOffset 15 endOffset 16 curOffset 24 d0000000000_00000000000000000023_00000_0000000001_00002| size 1048780 WTime 1763136182683
2025-11-14T16:03:02.683806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026193:3466] destroyed
2025-11-14T16:03:02.683820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026198:3469], now have 1 active actors on pipe
2025-11-14T16:03:02.683836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.683842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.683853Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.683866Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 15 partNo 1 count 8 size 7878063
2025-11-14T16:03:02.683874Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 23 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.683904Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|843f2aea-373a971c-f863a848-5b545c46_24 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.683911Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.683914Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.683918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.683920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.683924Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.683931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.683933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.683935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.683946Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.683982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|843f2aea-373a971c-f863a848-5b545c46_24"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
2025-11-14T16:03:02.685017Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.685432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026198:3469] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|843f2aea-373a971c-f863a848-5b545c46_24"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 24
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.692394Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.692775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026203:3472], now have 1 active actors on pipe
2025-11-14T16:03:02.693333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.693341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.693343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.693347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.693348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.698134Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.698153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.698242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 24 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.698309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 24 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.698321Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 24 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.698326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 24 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.698355Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.698371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.698375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.698384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.698387Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.698391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.698393Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.698400Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.698426Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 0
2025-11-14T16:03:02.698439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.698443Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 1
2025-11-14T16:03:02.698447Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.698450Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 2
2025-11-14T16:03:02.699155Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 24 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 1048790 count 1 nextOffset 25 batches 3
2025-11-14T16:03:02.699166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.699171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.699174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.699242Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 24,1 HeadOffset 24 endOffset 24 curOffset 25 d0000000000_00000000000000000024_00000_0000000001_00002? size 1048780 WTime 1763136182698
2025-11-14T16:03:02.699407Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.699419Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 24 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.703868Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 15 count 8 size 7878063 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.703881Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 23 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.703897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.703938Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 15 partno 1 count 8 parts 15 suffix '0' size 7878063
2025-11-14T16:03:02.703948Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 7 parts 16 suffix '0' size 8365339
2025-11-14T16:03:02.703952Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 2 suffix '124' size 1048780
2025-11-14T16:03:02.704099Z node 2 :PERSQUEUE DEBUG: partition.cpp:2289: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-14T16:03:02.704109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.704117Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 0 size 8365339 cause it's been evicted from L2. Actual L1 size: 12
2025-11-14T16:03:02.704121Z node 2 :PERSQUEUE NOTICE: read.h:372: [72075186224037892][PQCacheProxy]Have to remove new data from cache. Topic rt3.dc1--topic1, tablet id72075186224037892, cookie 0
2025-11-14T16:03:02.704771Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 24 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.704786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.704806Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.704810Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.704825Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 24, partNo: 0, Offset: 24 is stored on disk
2025-11-14T16:03:02.704832Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.704835Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 24, partNo: 1, Offset: 24 is stored on disk
2025-11-14T16:03:02.704840Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.704845Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 24, partNo: 2, Offset: 24 is stored on disk
2025-11-14T16:03:02.704866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.704872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.704874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.704876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.704878Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.704897Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.704926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.704941Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.704948Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.704949Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000016_00000_0000000001_00002?(+) to d0000000000_00000000000000000016_00000_0000000001_00002?(+)
2025-11-14T16:03:02.704951Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000017_00000_0000000001_00002?(+) to d0000000000_00000000000000000017_00000_0000000001_00002?(+)
2025-11-14T16:03:02.704953Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000018_00000_0000000001_00002?(+) to d0000000000_00000000000000000018_00000_0000000001_00002?(+)
2025-11-14T16:03:02.704955Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000019_00000_0000000001_00002?(+) to d0000000000_00000000000000000019_00000_0000000001_00002?(+)
2025-11-14T16:03:02.704957Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000020_00000_0000000001_00002?(+) to d0000000000_00000000000000000020_00000_0000000001_00002?(+)
2025-11-14T16:03:02.704959Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000021_00000_0000000001_00002?(+) to d0000000000_00000000000000000021_00000_0000000001_00002?(+)
2025-11-14T16:03:02.704960Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000022_00000_0000000001_00002?(+) to d0000000000_00000000000000000022_00000_0000000001_00002?(+)
2025-11-14T16:03:02.704962Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000023_00000_0000000001_00002?(+) to d0000000000_00000000000000000023_00000_0000000001_00002?(+)
2025-11-14T16:03:02.704964Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000015_00001_0000000001_00001|(+) to d0000000000_00000000000000000015_00001_0000000001_00001|(+)
2025-11-14T16:03:02.705008Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.705280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026203:3472] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 24
    Offset: 24
    PartNo: 0
    WriteTimestampMS: 1763136182697
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 6
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.705585Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 16 count 1 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.705595Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 17 count 1 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.705599Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 18 count 1 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.705602Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 19 count 1 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.705607Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 20 count 1 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.705610Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 21 count 1 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.705613Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 22 count 1 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.705618Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 23 count 1 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.705622Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 15 count 1 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.705635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.705645Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.705646Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.705650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.705652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.705653Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.705654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.705656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.705658Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.705657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.705663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.705663Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.705667Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.705673Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.705677Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.705682Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.705688Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 15 partno 1 count 1 parts 1 suffix '124' size 536753
2025-11-14T16:03:02.705992Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.706401Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026212:3479], now have 1 active actors on pipe
2025-11-14T16:03:02.706418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.706424Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.706519Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|12094816-eb7bb176-26b10e8a-5fdf7831_25 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.706530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.706532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.706536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.706538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.706542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.706548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.706551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.706553Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.706562Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.706571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.706710Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|12094816-eb7bb176-26b10e8a-5fdf7831_25"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
2025-11-14T16:03:02.706929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026212:3479] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|12094816-eb7bb176-26b10e8a-5fdf7831_25"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 25
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.713528Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.713933Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026216:3481], now have 1 active actors on pipe
2025-11-14T16:03:02.715029Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.715044Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.715117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 25 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.715177Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 25 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.715200Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 25 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.715208Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 25 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.715225Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.715244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.715251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.715257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.715259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.715263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.715265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.715269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.715302Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 0
2025-11-14T16:03:02.715317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.715325Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 1
2025-11-14T16:03:02.715328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.715332Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 2
2025-11-14T16:03:02.715566Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 25 partNo 2 FormedBlobsCount 0 NewHead: Offset 25 PartNo 0 PackedSize 1048790 count 1 nextOffset 26 batches 3
2025-11-14T16:03:02.715585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.715591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.715594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.715662Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 25,1 HeadOffset 25 endOffset 25 curOffset 26 d0000000000_00000000000000000025_00000_0000000001_00002? size 1048780 WTime 1763136182715
2025-11-14T16:03:02.715872Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.715892Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 25 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.718813Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 25 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.718836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.718845Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.718855Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.718861Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.718868Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 25, partNo: 0, Offset: 25 is stored on disk
2025-11-14T16:03:02.718876Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.718880Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 25, partNo: 1, Offset: 25 is stored on disk
2025-11-14T16:03:02.718896Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.718902Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 25, partNo: 2, Offset: 25 is stored on disk
2025-11-14T16:03:02.718925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.718932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.718934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.718936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.718938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.718945Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.718956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.719030Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 25
    Offset: 25
    PartNo: 0
    WriteTimestampMS: 1763136182714
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
2025-11-14T16:03:02.719311Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026216:3481] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.719759Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.720061Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026221:3484], now have 1 active actors on pipe
2025-11-14T16:03:02.720186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.720196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.720222Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ec4e8af0-86786fc-29ceef9b-88b9cfa2_26 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.720229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.720233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.720236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.720238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.720241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.720246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.720248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.720251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.720259Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.720269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.720515Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.720708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026221:3484] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ec4e8af0-86786fc-29ceef9b-88b9cfa2_26"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ec4e8af0-86786fc-29ceef9b-88b9cfa2_26"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 26
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.728197Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.728569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026224:3485], now have 1 active actors on pipe
2025-11-14T16:03:02.729616Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.729634Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.729732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 26 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.729784Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 26 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.729798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 26 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.729802Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 26 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.729817Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.729833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.729840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.729846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.729848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.729852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.729854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.729859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.729879Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 0
2025-11-14T16:03:02.729893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.729899Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 1
2025-11-14T16:03:02.729903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.729905Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 2
2025-11-14T16:03:02.730135Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 26 partNo 2 FormedBlobsCount 0 NewHead: Offset 26 PartNo 0 PackedSize 1048790 count 1 nextOffset 27 batches 3
2025-11-14T16:03:02.730149Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.730151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.730156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.730216Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 26,1 HeadOffset 26 endOffset 26 curOffset 27 d0000000000_00000000000000000026_00000_0000000001_00002? size 1048780 WTime 1763136182729
2025-11-14T16:03:02.730370Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.730385Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 26 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.732471Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 26 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.732494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.732506Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.732513Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.732518Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.732530Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 26, partNo: 0, Offset: 26 is stored on disk
2025-11-14T16:03:02.732534Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.732539Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 26, partNo: 1, Offset: 26 is stored on disk
2025-11-14T16:03:02.732544Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.732553Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 26, partNo: 2, Offset: 26 is stored on disk
2025-11-14T16:03:02.732579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.732592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.732598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.732601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.732603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.732609Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.732622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.732680Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.732945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026224:3485] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 26
    Offset: 26
    PartNo: 0
    WriteTimestampMS: 1763136182729
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.733561Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.733836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026229:3488], now have 1 active actors on pipe
2025-11-14T16:03:02.733908Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.733917Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.733952Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|e815c330-f53e087e-2eda987c-69da7617_27 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.733961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.733964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.733968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.733970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.733973Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.733979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.733984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.733986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.733994Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.734003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.734055Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|e815c330-f53e087e-2eda987c-69da7617_27"
    Status: Active
    SeqNo: 0
  }
}
2025-11-14T16:03:02.734254Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026229:3488] destroyed
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|e815c330-f53e087e-2eda987c-69da7617_27"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 27
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.739555Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.740004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026232:3489], now have 1 active actors on pipe
2025-11-14T16:03:02.741217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.741235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.741304Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 27 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.741361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 27 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.741370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 27 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.741373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 27 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.741392Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.741406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.741415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.741422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.741424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.741427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.741429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.741433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.741455Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 0
2025-11-14T16:03:02.741471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.741480Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 1
2025-11-14T16:03:02.741483Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.741485Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 2
2025-11-14T16:03:02.741712Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 27 partNo 2 FormedBlobsCount 0 NewHead: Offset 27 PartNo 0 PackedSize 1048790 count 1 nextOffset 28 batches 3
2025-11-14T16:03:02.741728Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.741731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.741736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.741794Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 27,1 HeadOffset 27 endOffset 27 curOffset 28 d0000000000_00000000000000000027_00000_0000000001_00002? size 1048780 WTime 1763136182741
2025-11-14T16:03:02.741935Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.741952Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 27 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.745805Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 27 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.745832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.745845Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.745852Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.745862Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.745870Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 27, partNo: 0, Offset: 27 is stored on disk
2025-11-14T16:03:02.745878Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.745882Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 27, partNo: 1, Offset: 27 is stored on disk
2025-11-14T16:03:02.745887Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.745891Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 27, partNo: 2, Offset: 27 is stored on disk
2025-11-14T16:03:02.745919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.745926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.745928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.745931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.745932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.745941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.745954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.746056Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.746398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026232:3489] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 27
    Offset: 27
    PartNo: 0
    WriteTimestampMS: 1763136182740
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.747076Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.747495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026238:3492], now have 1 active actors on pipe
2025-11-14T16:03:02.747591Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.747601Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.747631Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|97d42aee-5f01e9bb-5bb6d3-2646be0c_28 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.747642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.747645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.747650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.747652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.747655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.747662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.747664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.747667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.747677Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.747689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
2025-11-14T16:03:02.747746Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|97d42aee-5f01e9bb-5bb6d3-2646be0c_28"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|97d42aee-5f01e9bb-5bb6d3-2646be0c_28"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 28
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.751117Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026238:3492] destroyed
2025-11-14T16:03:02.753611Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.754016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026241:3493], now have 1 active actors on pipe
2025-11-14T16:03:02.755122Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.755138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.755224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 28 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.755276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 28 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.755289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 28 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.755293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 28 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.755310Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.755339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.755348Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.755358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.755365Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.755368Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.755371Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.755376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.755400Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 0
2025-11-14T16:03:02.755415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.755422Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 1
2025-11-14T16:03:02.755424Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.755427Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 2
2025-11-14T16:03:02.755640Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 28 partNo 2 FormedBlobsCount 0 NewHead: Offset 28 PartNo 0 PackedSize 1048790 count 1 nextOffset 29 batches 3
2025-11-14T16:03:02.755654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.755658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.755661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.755720Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 28,1 HeadOffset 28 endOffset 28 curOffset 29 d0000000000_00000000000000000028_00000_0000000001_00002? size 1048780 WTime 1763136182755
2025-11-14T16:03:02.755895Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.755913Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 28 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.758864Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 28 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.758887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.758900Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.758908Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.758917Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.758925Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 28, partNo: 0, Offset: 28 is stored on disk
2025-11-14T16:03:02.758931Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.758935Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 28, partNo: 1, Offset: 28 is stored on disk
2025-11-14T16:03:02.758940Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.758944Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 28, partNo: 2, Offset: 28 is stored on disk
2025-11-14T16:03:02.758968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.758978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.758980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.758982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.758985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.758993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.759005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.759106Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.759499Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026241:3493] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 28
    Offset: 28
    PartNo: 0
    WriteTimestampMS: 1763136182755
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.760267Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.760669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026246:3496], now have 1 active actors on pipe
2025-11-14T16:03:02.760772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.760785Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.760830Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|e14af0a5-278b24d1-214e98a0-4bffc90e_29 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.760842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.760849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.760855Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.760858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.760861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.760874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.760881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.760883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.760896Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.760923Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.760980Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.761222Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026246:3496] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|e14af0a5-278b24d1-214e98a0-4bffc90e_29"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|e14af0a5-278b24d1-214e98a0-4bffc90e_29"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 29
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.767045Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.767449Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026249:3497], now have 1 active actors on pipe
2025-11-14T16:03:02.768444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.768464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.768534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 29 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.768610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 29 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.768626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 29 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.768630Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 29 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.768649Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.768671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.768678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.768688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.768696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.768699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.768702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.768705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.768732Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 0
2025-11-14T16:03:02.768750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.768758Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 1
2025-11-14T16:03:02.768761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.768765Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 2
2025-11-14T16:03:02.769028Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 29 partNo 2 FormedBlobsCount 0 NewHead: Offset 29 PartNo 0 PackedSize 1048790 count 1 nextOffset 30 batches 3
2025-11-14T16:03:02.769049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.769051Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.769055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.769124Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 29,1 HeadOffset 29 endOffset 29 curOffset 30 d0000000000_00000000000000000029_00000_0000000001_00002? size 1048780 WTime 1763136182769
2025-11-14T16:03:02.769323Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.769341Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 29 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.773186Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 29 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.773216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.773238Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.773239Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.773249Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.773260Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 29, partNo: 0, Offset: 29 is stored on disk
2025-11-14T16:03:02.773268Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.773273Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 29, partNo: 1, Offset: 29 is stored on disk
2025-11-14T16:03:02.773278Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.773282Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 29, partNo: 2, Offset: 29 is stored on disk
2025-11-14T16:03:02.773310Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.773317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.773319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.773322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.773324Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.773332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.773344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.773443Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 29
    Offset: 29
    PartNo: 0
    WriteTimestampMS: 1763136182768
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.774033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026249:3497] destroyed
2025-11-14T16:03:02.774610Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.775034Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026254:3500], now have 1 active actors on pipe
2025-11-14T16:03:02.775137Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.775149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.775193Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|3e23ee7d-8a8ce444-599117b9-c8c5fff6_30 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.775204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.775207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.775212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.775213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.775217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.775226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.775228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.775231Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.775243Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.775259Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.775311Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.775568Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026254:3500] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|3e23ee7d-8a8ce444-599117b9-c8c5fff6_30"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|3e23ee7d-8a8ce444-599117b9-c8c5fff6_30"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 30
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.781788Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.782342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026257:3501], now have 1 active actors on pipe
2025-11-14T16:03:02.783662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.783682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.783775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 30 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.783856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 30 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.783873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 30 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.783878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 30 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.783904Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.783931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.783939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.783947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.783950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.783954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.783956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.783961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.783989Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 0
2025-11-14T16:03:02.784009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.784017Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 1
2025-11-14T16:03:02.784020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.784023Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 2
2025-11-14T16:03:02.784280Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 30 partNo 2 FormedBlobsCount 0 NewHead: Offset 30 PartNo 0 PackedSize 1048790 count 1 nextOffset 31 batches 3
2025-11-14T16:03:02.784298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.784300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.784304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.784371Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 30,1 HeadOffset 30 endOffset 30 curOffset 31 d0000000000_00000000000000000030_00000_0000000001_00002? size 1048780 WTime 1763136182783
2025-11-14T16:03:02.784514Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.784533Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 30 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.787560Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 30 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.787582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.787593Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.787602Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.787607Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.787620Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 30, partNo: 0, Offset: 30 is stored on disk
2025-11-14T16:03:02.787624Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.787628Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 30, partNo: 1, Offset: 30 is stored on disk
2025-11-14T16:03:02.787639Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.787649Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 30, partNo: 2, Offset: 30 is stored on disk
2025-11-14T16:03:02.787676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.787682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.787686Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.787688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.787690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.787696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.787709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.787797Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.788105Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026257:3501] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 30
    Offset: 30
    PartNo: 0
    WriteTimestampMS: 1763136182783
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.789000Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.789535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026262:3504], now have 1 active actors on pipe
2025-11-14T16:03:02.789685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.789699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.789731Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|7d09c82d-79b2b342-820ef6b-4121bafd_31 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.789743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.789751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.789756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.789758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.789762Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.789770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.789772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.789791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.789807Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.789821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.789881Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.790112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026262:3504] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|7d09c82d-79b2b342-820ef6b-4121bafd_31"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|7d09c82d-79b2b342-820ef6b-4121bafd_31"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 31
    Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:02.793727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.793740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.793743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.793746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.793748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.796573Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.797113Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026265:3505], now have 1 active actors on pipe
2025-11-14T16:03:02.798613Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.798632Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.798708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 31 partNo : 0 messageNo: 0 size: 511959
2025-11-14T16:03:02.798773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 31 partNo : 1 messageNo: 0 size: 511959
2025-11-14T16:03:02.798791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1977: [PQ: 72075186224037892] got client PART message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 31 partNo : 2 messageNo: 0 size: 24658
2025-11-14T16:03:02.798795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 31 partNo : 2 messageNo: 0 size 24658 offset: -1
2025-11-14T16:03:02.798821Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.798842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.798850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.798860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.798863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.798866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 3, PendingWrites: 0
2025-11-14T16:03:02.798868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.798873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.798901Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 0
2025-11-14T16:03:02.798928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.798931Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 1
2025-11-14T16:03:02.798934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.798936Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 2
2025-11-14T16:03:02.799158Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 31 partNo 2 FormedBlobsCount 0 NewHead: Offset 31 PartNo 0 PackedSize 1048790 count 1 nextOffset 32 batches 3
2025-11-14T16:03:02.799174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 3
2025-11-14T16:03:02.799177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (3)
2025-11-14T16:03:02.799180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.799240Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 31,1 HeadOffset 31 endOffset 31 curOffset 32 d0000000000_00000000000000000031_00000_0000000001_00002? size 1048780 WTime 1763136182798
2025-11-14T16:03:02.799401Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.799422Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 31 partNo 0 count 1 size 1048780
2025-11-14T16:03:02.802726Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 31 count 1 size 1048780 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.802752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.802773Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.802775Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 1048597 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.802781Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.802789Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 31, partNo: 0, Offset: 31 is stored on disk
2025-11-14T16:03:02.802795Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.802803Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 31, partNo: 1, Offset: 31 is stored on disk
2025-11-14T16:03:02.802813Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.802817Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 31, partNo: 2, Offset: 31 is stored on disk
2025-11-14T16:03:02.802848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.802857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.802859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.802862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.802864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.802874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000024_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.802879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000025_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.802882Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000026_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.802885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000027_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.802888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000028_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.802890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000029_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.802893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000030_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.802895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000031_00000_0000000001_00002? (1048780)
2025-11-14T16:03:02.802898Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 8 keys were taken away. Let's read 8390240 bytes
2025-11-14T16:03:02.802914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.802925Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 8 blobs
2025-11-14T16:03:02.802928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000024_00000_0000000001_00002?
2025-11-14T16:03:02.802937Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000025_00000_0000000001_00002?
2025-11-14T16:03:02.802948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000026_00000_0000000001_00002?
2025-11-14T16:03:02.802965Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000027_00000_0000000001_00002?
2025-11-14T16:03:02.802977Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000028_00000_0000000001_00002?
2025-11-14T16:03:02.802986Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.803006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000029_00000_0000000001_00002?
2025-11-14T16:03:02.803019Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000030_00000_0000000001_00002?
2025-11-14T16:03:02.803028Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000031_00000_0000000001_00002?
2025-11-14T16:03:02.803038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 8 blobs for compaction
2025-11-14T16:03:02.803046Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 24 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.803055Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 25 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.803059Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 26 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.803063Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 27 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.803066Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 28 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.803070Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 29 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.803074Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 30 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.803077Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 31 partno 0 count 1 parts_count 2 source 1 size 1048780 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.803099Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 8 blobs are from cache.
2025-11-14T16:03:02.803102Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.803106Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.803110Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.803113Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.803116Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.803120Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.803125Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.803127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-14T16:03:02.803134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/8] d0000000000_00000000000000000024_00000_0000000001_00002?
2025-11-14T16:03:02.803135Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 2 suffix '63'
2025-11-14T16:03:02.803136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.803139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000024_00000_0000000001_00002?, parameters.CurOffset=24
2025-11-14T16:03:02.803257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 24.0/3
2025-11-14T16:03:02.803270Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 0
2025-11-14T16:03:02.803275Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 24:0 appended
CallPersQueueGRPC response:
2025-11-14T16:03:02.803419Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 24.1/3
2025-11-14T16:03:02.803427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 1
2025-11-14T16:03:02.803431Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 24:1 appended
2025-11-14T16:03:02.803447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 24.2/3
2025-11-14T16:03:02.803454Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 2
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 31
    Offset: 31
    PartNo: 0
    WriteTimestampMS: 1763136182798
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
>>>>> 1
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdRead {
    ClientId: "user"
    Offset: 0
    Count: 16
    ReadTimestampMs: 0
  }
}
2025-11-14T16:03:02.803673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 24 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 1048790 count 1 nextOffset 25 batches 3
2025-11-14T16:03:02.803683Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 24:2 appended
2025-11-14T16:03:02.803687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/8] d0000000000_00000000000000000025_00000_0000000001_00002?
2025-11-14T16:03:02.803689Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.803691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000025_00000_0000000001_00002?, parameters.CurOffset=25
2025-11-14T16:03:02.803821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 25.0/3
2025-11-14T16:03:02.803832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 0
2025-11-14T16:03:02.803835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 25:0 appended
2025-11-14T16:03:02.803970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 25.1/3
2025-11-14T16:03:02.803979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 1
2025-11-14T16:03:02.803982Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 25:1 appended
2025-11-14T16:03:02.803995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 25.2/3
2025-11-14T16:03:02.803999Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 2
2025-11-14T16:03:02.804163Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 25 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 2097545 count 2 nextOffset 26 batches 5
2025-11-14T16:03:02.804176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 25:2 appended
2025-11-14T16:03:02.804180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/8] d0000000000_00000000000000000026_00000_0000000001_00002?
2025-11-14T16:03:02.804181Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.804184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000026_00000_0000000001_00002?, parameters.CurOffset=26
2025-11-14T16:03:02.804314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 26.0/3
2025-11-14T16:03:02.804323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 0
2025-11-14T16:03:02.804326Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 26:0 appended
2025-11-14T16:03:02.804461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 26.1/3
2025-11-14T16:03:02.804470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 1
2025-11-14T16:03:02.804472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 26:1 appended
2025-11-14T16:03:02.804485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 26.2/3
2025-11-14T16:03:02.804487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 2
2025-11-14T16:03:02.804496Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.804658Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 26 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 3146300 count 3 nextOffset 27 batches 7
2025-11-14T16:03:02.804668Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 26:2 appended
2025-11-14T16:03:02.804671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/8] d0000000000_00000000000000000027_00000_0000000001_00002?
2025-11-14T16:03:02.804673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.804675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000027_00000_0000000001_00002?, parameters.CurOffset=27
2025-11-14T16:03:02.804818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 27.0/3
2025-11-14T16:03:02.804826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 0
2025-11-14T16:03:02.804829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 27:0 appended
2025-11-14T16:03:02.805330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 27.1/3
2025-11-14T16:03:02.805343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 1
2025-11-14T16:03:02.805345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 27:1 appended
2025-11-14T16:03:02.805358Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 27.2/3
2025-11-14T16:03:02.805363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 2
2025-11-14T16:03:02.805531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 27 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 4195055 count 4 nextOffset 28 batches 9
2025-11-14T16:03:02.805542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 27:2 appended
2025-11-14T16:03:02.805545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/8] d0000000000_00000000000000000028_00000_0000000001_00002?
2025-11-14T16:03:02.805547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.805550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000028_00000_0000000001_00002?, parameters.CurOffset=28
2025-11-14T16:03:02.805687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 28.0/3
2025-11-14T16:03:02.805698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 0
2025-11-14T16:03:02.805703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 28:0 appended
2025-11-14T16:03:02.805835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 28.1/3
2025-11-14T16:03:02.805843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 1
2025-11-14T16:03:02.805845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 28:1 appended
2025-11-14T16:03:02.805856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 28.2/3
2025-11-14T16:03:02.805858Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 2
2025-11-14T16:03:02.806028Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 28 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 5243810 count 5 nextOffset 29 batches 11
2025-11-14T16:03:02.806031Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 28:2 appended
2025-11-14T16:03:02.806034Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/8] d0000000000_00000000000000000029_00000_0000000001_00002?
2025-11-14T16:03:02.806036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.806039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000029_00000_0000000001_00002?, parameters.CurOffset=29
2025-11-14T16:03:02.806180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 29.0/3
2025-11-14T16:03:02.806191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 0
2025-11-14T16:03:02.806194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 29:0 appended
2025-11-14T16:03:02.806327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 29.1/3
2025-11-14T16:03:02.806335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 1
2025-11-14T16:03:02.806337Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 29:1 appended
2025-11-14T16:03:02.806348Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 29.2/3
2025-11-14T16:03:02.806354Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 2
2025-11-14T16:03:02.806524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 29 partNo 2 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 6292565 count 6 nextOffset 30 batches 13
2025-11-14T16:03:02.806536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 29:2 appended
2025-11-14T16:03:02.806539Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/8] d0000000000_00000000000000000030_00000_0000000001_00002?
2025-11-14T16:03:02.806541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.806543Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000030_00000_0000000001_00002?, parameters.CurOffset=30
2025-11-14T16:03:02.806679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 30.0/3
2025-11-14T16:03:02.806688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 0
2025-11-14T16:03:02.806690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 30:0 appended
2025-11-14T16:03:02.806835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 30.1/3
2025-11-14T16:03:02.806843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 1
2025-11-14T16:03:02.806845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 30:1 appended
2025-11-14T16:03:02.806856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 30.2/3
2025-11-14T16:03:02.806859Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 2
2025-11-14T16:03:02.817624Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 30 partNo 2 result is x0000000000_00000000000000000023_00000_0000000007_00016 size 8365364
2025-11-14T16:03:02.817650Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1104: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000023_00000_0000000007_00016 new key d0000000000_00000000000000000023_00000_0000000007_00016 size 8365364 WTime 1763136182817
2025-11-14T16:03:02.817691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 30 partNo 2 FormedBlobsCount 1 NewHead: Offset 30 PartNo 2 PackedSize 24736 count 1 nextOffset 31 batches 1
2025-11-14T16:03:02.817698Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 30:2 appended
2025-11-14T16:03:02.817705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/8] d0000000000_00000000000000000031_00000_0000000001_00002?
2025-11-14T16:03:02.817708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.817713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000031_00000_0000000001_00002?, parameters.CurOffset=31
2025-11-14T16:03:02.817844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 31.0/3
2025-11-14T16:03:02.817857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 0
2025-11-14T16:03:02.817860Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 31:0 appended
2025-11-14T16:03:02.817997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 31.1/3
2025-11-14T16:03:02.818006Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 1
2025-11-14T16:03:02.818009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 31:1 appended
2025-11-14T16:03:02.818024Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 31.2/3
2025-11-14T16:03:02.818026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 2
2025-11-14T16:03:02.818212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 31 partNo 2 FormedBlobsCount 0 NewHead: Offset 30 PartNo 2 PackedSize 1073491 count 2 nextOffset 32 batches 3
2025-11-14T16:03:02.818224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 31:2 appended
2025-11-14T16:03:02.818273Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 30,2 HeadOffset 23 endOffset 24 curOffset 32 d0000000000_00000000000000000030_00002_0000000002_00002| size 1073481 WTime 1763136182818
2025-11-14T16:03:02.818445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026265:3505] destroyed
2025-11-14T16:03:02.818459Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026272:3509], now have 1 active actors on pipe
2025-11-14T16:03:02.818484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.818493Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.818508Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.818527Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 23 partNo 0 count 7 size 8365364
2025-11-14T16:03:02.818538Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 30 partNo 2 count 2 size 1073481
2025-11-14T16:03:02.818620Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 4 Topic 'rt3.dc1--topic1' partition 0 user user offset 0 partno 0 count 16 size 26214400 endOffset 32 max time lag 0ms effective offset 0
2025-11-14T16:03:02.818826Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 4 added 3 blobs, size 25145475 count 24 last offset 15, current partition end offset: 32
2025-11-14T16:03:02.818842Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 4. Send blob request.
2025-11-14T16:03:02.818855Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:474: No blob in L1. Partition 0 offset 0 partno 0 count 7 parts_count 16 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.818860Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 7 partno 2 count 8 parts_count 15 source 1 size 7878063 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.818863Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 15 partno 1 count 8 parts_count 15 source 1 size 7878063 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.818878Z node 2 :PERSQUEUE DEBUG: read.h:142: [72075186224037892][PQCacheProxy]Reading cookie 4. Have to read 1 of 3 from KV
2025-11-14T16:03:02.818910Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 7 partno 2 count 8 parts 15 suffix '0'
2025-11-14T16:03:02.818923Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 15 partno 1 count 8 parts 15 suffix '0'
2025-11-14T16:03:02.818930Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:223: PQ Cache (L2). Missed blob. tabletId '72075186224037892' partition 0 offset 0 partno 0 count 7 parts_count 16
2025-11-14T16:03:02.831982Z node 2 :PERSQUEUE DEBUG: read.h:178: [72075186224037892][PQCacheProxy]Got results. 1 of 3 from KV. Status 1
2025-11-14T16:03:02.832010Z node 2 :PERSQUEUE DEBUG: read.h:195: [72075186224037892][PQCacheProxy]Got results. result 0 from KV. Status 0
2025-11-14T16:03:02.832026Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:408: Prefetched blob in L1. Partition 0 offset 0 count 7 size 8365339 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.832098Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 2 suffix '124' size 1048780
2025-11-14T16:03:02.832118Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 7 parts 16 suffix '0' size 8365339
2025-11-14T16:03:02.832123Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 3 blobs
2025-11-14T16:03:02.832364Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.832588Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.832876Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-14T16:03:02.833117Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 1 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.833368Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 1 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-14T16:03:02.833556Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 2 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.833974Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 2 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-14T16:03:02.834168Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 3 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.834404Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 3 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-14T16:03:02.834605Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 4 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.834848Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 4 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-14T16:03:02.835041Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 5 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.835301Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 5 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-14T16:03:02.835492Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 6 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.835537Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 6 totakecount 7 count 1 size 24704 from pos 0 cbcount 1
2025-11-14T16:03:02.835905Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 7 totakecount 7 count 0 size 1024007 from pos 0 cbcount 2
2025-11-14T16:03:02.836000Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 7 totakecount 8 count 1 size 24704 from pos 0 cbcount 1
2025-11-14T16:03:02.836179Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 8 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.836387Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 8 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.836609Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 8 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-14T16:03:02.836798Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 9 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.837117Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 9 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-14T16:03:02.837334Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 10 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.837556Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 10 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-14T16:03:02.837777Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 11 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.838061Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 11 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-14T16:03:02.838254Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 12 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.838469Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 12 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-14T16:03:02.838654Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 13 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.838874Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 13 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-14T16:03:02.839084Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 14 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.839105Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 14 totakecount 8 count 1 size 24704 from pos 0 cbcount 1
2025-11-14T16:03:02.839304Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 15 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.839937Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 15 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.839957Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 15 totakecount 8 count 1 size 24704 from pos 0 cbcount 1
2025-11-14T16:03:02.840030Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 23 size 1048780 cause it's been evicted from L2. Actual L1 size: 11
2025-11-14T16:03:02.840039Z node 2 :PERSQUEUE NOTICE: read.h:372: [72075186224037892][PQCacheProxy]Have to remove new data from cache. Topic rt3.dc1--topic1, tablet id72075186224037892, cookie 0
2025-11-14T16:03:02.840116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.860383Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 23 count 7 size 8365364 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.860404Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 30 count 2 size 1073481 actorID [2:7572612242605387853:2363]
2025-11-14T16:03:02.860551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.860581Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.860598Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.860604Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.860607Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.860610Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.860614Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.860618Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.860621Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 7 parts 16 suffix '0' size 8365364
2025-11-14T16:03:02.860630Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 2 suffix '63' size 1048780
2025-11-14T16:03:02.860634Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 7 partno 2 count 8 parts 15 suffix '0' size 7878063
2025-11-14T16:03:02.860636Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 30 partno 2 count 2 parts 2 suffix '124' size 1073481
2025-11-14T16:03:02.860804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.860817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.860820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.860823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.860825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.860863Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.860881Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 29 size 1048780 cause it's been evicted from L2. Actual L1 size: 12
2025-11-14T16:03:02.860889Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 26 size 1048780 cause it's been evicted from L2. Actual L1 size: 11
2025-11-14T16:03:02.860892Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 30 size 1048780 cause it's been evicted from L2. Actual L1 size: 10
2025-11-14T16:03:02.860895Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 25 size 1048780 cause it's been evicted from L2. Actual L1 size: 9
2025-11-14T16:03:02.860912Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 27 size 1048780 cause it's been evicted from L2. Actual L1 size: 8
2025-11-14T16:03:02.860915Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 31 size 1048780 cause it's been evicted from L2. Actual L1 size: 7
2025-11-14T16:03:02.860918Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 7 size 7878063 cause it's been evicted from L2. Actual L1 size: 6
2025-11-14T16:03:02.860921Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 24 size 1048780 cause it's been evicted from L2. Actual L1 size: 5
2025-11-14T16:03:02.860924Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 28 size 1048780 cause it's been evicted from L2. Actual L1 size: 4
2025-11-14T16:03:02.860930Z node 2 :PERSQUEUE NOTICE: read.h:372: [72075186224037892][PQCacheProxy]Have to remove new data from cache. Topic rt3.dc1--topic1, tablet id72075186224037892, cookie 0
2025-11-14T16:03:02.860964Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.860978Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000024_00000_0000000001_00002?(+) to d0000000000_00000000000000000024_00000_0000000001_00002?(+)
2025-11-14T16:03:02.860986Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000025_00000_0000000001_00002?(+) to d0000000000_00000000000000000025_00000_0000000001_00002?(+)
2025-11-14T16:03:02.860989Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000026_00000_0000000001_00002?(+) to d0000000000_00000000000000000026_00000_0000000001_00002?(+)
2025-11-14T16:03:02.860993Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000027_00000_0000000001_00002?(+) to d0000000000_00000000000000000027_00000_0000000001_00002?(+)
2025-11-14T16:03:02.860996Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000028_00000_0000000001_00002?(+) to d0000000000_00000000000000000028_00000_0000000001_00002?(+)
2025-11-14T16:03:02.860998Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000029_00000_0000000001_00002?(+) to d0000000000_00000000000000000029_00000_0000000001_00002?(+)
2025-11-14T16:03:02.861001Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000030_00000_0000000001_00002?(+) to d0000000000_00000000000000000030_00000_0000000001_00002?(+)
2025-11-14T16:03:02.861004Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000031_00000_0000000001_00002?(+) to d0000000000_00000000000000000031_00000_0000000001_00002?(+)
2025-11-14T16:03:02.861007Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000023_00000_0000000001_00002|(+) to d0000000000_00000000000000000023_00000_0000000001_00002|(+)
2025-11-14T16:03:02.869317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.869354Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.869367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.869372Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.869375Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.869378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.869380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.869392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.877193Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.887901Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026272:3509] destroyed
2025-11-14T16:03:02.894052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.894075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.894079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.894083Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.894086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdReadResult {
    MaxOffset: 32
    Result {
      Offset: 0
      Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
>>>>> 2
CallPersQueueGRPC request to localhost:27509
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdRead {
    ClientId: "user"
    Offset: 16
    Count: 16
    ReadTimestampMs: 0
  }
}
2025-11-14T16:03:02.967746Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.968968Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612242852026289:3523], now have 1 active actors on pipe
2025-11-14T16:03:02.969015Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.969026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.969073Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 5 Topic 'rt3.dc1--topic1' partition 0 user user offset 16 partno 0 count 16 size 26214400 endOffset 32 max time lag 0ms effective offset 16
2025-11-14T16:03:02.969398Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 5 added 2 blobs, size 9438782 count 16 last offset 23, current partition end offset: 32
2025-11-14T16:03:02.969413Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 5. Send blob request.
2025-11-14T16:03:02.969432Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 15 partno 1 count 8 parts_count 15 source 1 size 7878063 accessed 1 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.969438Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 23 partno 0 count 7 parts_count 16 source 1 size 8365364 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.969454Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 5. All 2 blobs are from cache.
2025-11-14T16:03:02.969490Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 15 partno 1 count 8 parts 15 suffix '0'
2025-11-14T16:03:02.969501Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 2 blobs
2025-11-14T16:03:02.969503Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 7 parts 16 suffix '0'
2025-11-14T16:03:02.969866Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 16 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.970128Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 16 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.970331Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 16 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-14T16:03:02.970628Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 17 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.970828Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 17 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-14T16:03:02.971141Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 18 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.971334Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 18 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-14T16:03:02.971657Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 19 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.971853Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 19 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-14T16:03:02.972030Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 20 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.972229Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 20 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-14T16:03:02.972591Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 21 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.972784Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 21 totakecount 8 count 1 size 536706 from pos 0 cbcount 2
2025-11-14T16:03:02.973043Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 22 totakecount 8 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.973072Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 22 totakecount 8 count 1 size 24704 from pos 0 cbcount 1
2025-11-14T16:03:02.973314Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 23 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.973493Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 23 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.973525Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 23 totakecount 7 count 1 size 24704 from pos 0 cbcount 1
2025-11-14T16:03:02.973692Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 24 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.973871Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 24 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.974064Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 24 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-14T16:03:02.974239Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 25 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.974435Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 25 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-14T16:03:02.974604Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 26 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.974800Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 26 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-14T16:03:02.974976Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 27 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.975177Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 27 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-14T16:03:02.975358Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 28 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.975556Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 28 totakecount 7 count 1 size 536706 from pos 0 cbcount 2
2025-11-14T16:03:02.975742Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 29 totakecount 7 count 0 size 512005 from pos 0 cbcount 1
2025-11-14T16:03:02.975770Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 29 totakecount 7 count 1 size 24704 from pos 0 cbcount 1
2025-11-14T16:03:02.976488Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 30 totakecount 7 count 0 size 1024007 from pos 0 cbcount 2
2025-11-14T16:03:02.976605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.994408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.994443Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.994446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.994450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.994453Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.998397Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:03.009399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612242852026289:3523] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdReadResult {
    MaxOffset: 32
    Result {
      Offset: 16
      Data: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx...
(TRUNCATED)
2025-11-14T16:03:03.094803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.094823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.094827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.094835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.094839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.195155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.195209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.195212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.195215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.195219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.295509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.295533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.295537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.295541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.295545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
[[bad]]assertion failed at ydb/services/persqueue_v1/persqueue_ut.cpp:3269, virtual void NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TTestCaseCache::Execute_(NUnitTest::TTestContext &): (info0.BlobsFromCache == 1) failed: (2 != 1) [[rst]]
[[alt1]]TBackTrace::Capture()+28 (0x14F5D62C)
NUnitTest::NPrivate::RaiseError(char const*, TBasicString<char, std::__y1::char_traits<char>> const&, bool)+128 (0x151197B0)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TTestCaseCache::Execute_(NUnitTest::TTestContext&)+4375 (0x14C87B77)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TCurrentTest::Execute()::'lambda'()::operator()() const+71 (0x14D732D7)
NUnitTest::TTestBase::Run(std::__y1::function<void ()>, TBasicString<char, std::__y1::char_traits<char>> const&, char const*, bool)+120 (0x1511B988)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TCurrentTest::Execute()+436 (0x14D72BD4)
NUnitTest::TTestFactory::Execute()+817 (0x1511C0D1)
NUnitTest::RunMain(int, char**)+3229 (0x1512E53D)
??+0 (0x7F02BAB96D90)
__libc_start_main+128 (0x7F02BAB96E40)
_start+41 (0x13D70029)
[[rst]]
2025-11-14T16:02:56.183894Z node 1 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[1:7572612214870145289:2088];send_to=[0:7307199536658146131:7762515];
2025-11-14T16:02:56.184001Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
2025-11-14T16:02:56.189273Z node 1 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-11-14T16:02:56.190042Z node 2 :METADATA_PROVIDER WARN: log.cpp:841: fline=table_exists.cpp:54;actor=TTableExistsActor;event=undelivered;self_id=[2:7572612215353570463:2154];send_to=[0:7307199536658146131:7762515];
2025-11-14T16:02:56.190147Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=scheme_cache_undelivered_message;
test_client.cpp: SetPath # /home/runner/.ya/build/build_root/bku8/000126/r3tmp/tmpSxkEqO/pdisk_1.dat
2025-11-14T16:02:56.191900Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:123: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, got undelivered to scheme cache: ActorUnknown
2025-11-14T16:02:56.192032Z node 2 :PQ_READ_PROXY DEBUG: caching_service.cpp:44: Direct read cache: : Created
2025-11-14T16:02:56.236592Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: LookupError, path: Root/.metadata/script_executions
2025-11-14T16:02:56.249638Z node 1 :IMPORT WARN: schemeshard_import.cpp:307: Table profiles were not loaded
2025-11-14T16:02:56.264470Z node 2 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
TServer::EnableGrpc on GrpcPort 2468, node 1
2025-11-14T16:02:56.270772Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:190: distributable config is empty, broken or outdated, will use file: /home/runner/.ya/build/build_root/bku8/000126/r3tmp/yandexyCn5Vj.tmp
2025-11-14T16:02:56.270786Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:196: will try to initialize from file: /home/runner/.ya/build/build_root/bku8/000126/r3tmp/yandexyCn5Vj.tmp
2025-11-14T16:02:56.270881Z node 1 :NET_CLASSIFIER WARN: net_classifier.cpp:202: successfully initialized from file: /home/runner/.ya/build/build_root/bku8/000126/r3tmp/yandexyCn5Vj.tmp
2025-11-14T16:02:56.270940Z node 1 :NET_CLASSIFIER ERROR: net_classifier.cpp:228: got bad distributable configuration
2025-11-14T16:02:56.275424Z INFO: TTestServer started on Port 6907 GrpcPort 2468
2025-11-14T16:02:56.287186Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-11-14T16:02:56.287221Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-11-14T16:02:56.289505Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(1, (0,0,0,0)) VolatileState: Connecting -> Connected
TClient is connected to server localhost:6907
PQClient connected to localhost:2468
WaitRootIsUp 'Root'...
TClient::Ls request: Root
TClient::Ls response: 2025-11-14T16:02:56.295865Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Unknown -> Disconnected
2025-11-14T16:02:56.295888Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Disconnected -> Connecting
2025-11-14T16:02:56.296996Z node 1 :HIVE WARN: hive_impl.cpp:811: HIVE#72057594037968897 Handle TEvInterconnect::TEvNodeConnected, NodeId 2 Cookie 2
2025-11-14T16:02:56.297300Z node 1 :HIVE WARN: node_info.cpp:25: HIVE#72057594037968897 Node(2, (0,0,0,0)) VolatileState: Connecting -> Connected
Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "Root"
    PathId: 1
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 1
    CreateStep: 0
    ParentPathId: 1
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 1
      SubDomainVersion: 0
      SecurityStateVersion: 0
    }
    ChildrenExist: false
  }
  Children {
    Name: ".sys"
    PathId: 18446744073709551615
    SchemeshardId: 72057594046644480
    PathType: EPathTypeDir
    CreateFinished: true
    CreateTxId: 0
    CreateStep: 0
    ParentPathId: 18446744073709551615
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 0
      Pl...
(TRUNCATED)
WaitRootIsUp 'Root' success.
2025-11-14T16:02:56.304746Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpAlterSubDomain, opId: 281474976720657:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_alter_subdomain.cpp:311)
waiting...
waiting...
2025-11-14T16:02:56.321751Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720659:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
waiting...
waiting...
waiting...
2025-11-14T16:02:56.478464Z node 1 :KQP_PROXY WARN: kqp_finalize_script_service.cpp:135: [ScriptExecutions] [TKqpFinalizeScriptService] Failed to check script execution tables existence, scheme status: PathErrorUnknown, path: Root/.metadata/script_executions
2025-11-14T16:02:56.614928Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7572612215353570756:2303], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-14T16:02:56.614934Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7572612215353570724:2300], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-14T16:02:56.614951Z node 2 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-14T16:02:56.615127Z node 2 :KQP_WORKLOAD_SERVICE WARN: scheme_actors.cpp:225: [WorkloadService] [TPoolFetcherActor] ActorId: [2:7572612215353570762:2305], DatabaseId: /Root, PoolId: default, Failed to fetch pool info, NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-14T16:02:56.615154Z node 2 :KQP_WORKLOAD_SERVICE WARN: kqp_workload_service.cpp:263: [WorkloadService] [Service] Failed to fetch pool default, DatabaseId: /Root, status: NOT_FOUND, issues: { <main>: Error: Resource pool default not found or you don't have access permissions }
2025-11-14T16:02:56.616356Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateResourcePool, opId: 281474976715657:3, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:179)
2025-11-14T16:02:56.622630Z node 2 :KQP_WORKLOAD_SERVICE WARN: helpers.h:73: [WorkloadService] [TPoolCreatorActor] ActorId: [2:7572612215353570761:2304], DatabaseId: /Root, PoolId: default, Scheduled retry for error: { <main>: Error: Transaction 281474976715657 completed, doublechecking }
2025-11-14T16:02:56.682148Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720662:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
2025-11-14T16:02:56.682465Z node 1 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [1:7572612214870146333:2331], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:3:16: Error: At function: KiReadTable!
        <main>:3:16: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Cluster]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-11-14T16:02:56.682635Z node 1 :KQP_SESSION WARN: kqp_session_actor.cpp:2517: SessionId: ydb://session/3?node_id=1&id=ZmE0YzgyNjMtZWNhNzA0YmMtYWIyZjg0ODgtN2ZiNWI1ZjQ=, ActorId: [1:7572612214870146283:2323], ActorState: ExecuteState, TraceId: 01ka1hp0fxa2n8krgssm08rydj, ReplyQueryCompileError, status: SCHEME_ERROR, issues: { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } }, remove tx with tx_id: 
2025-11-14T16:02:56.683137Z node 1 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 3 column: 16 } message: "At function: KiReadTable!" end_position { row: 3 column: 16 } severity: 1 issues { position { row: 3 column: 16 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Cluster]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 3 column: 16 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-11-14T16:02:56.702277Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720663:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
2025-11-14T16:02:56.708494Z node 2 :TX_PROXY ERROR: schemereq.cpp:593: Actor# [2:7572612215353570816:2190] txid# 281474976715658, issues: { message: "Check failed: path: \'/Root/.metadata/workload_manager/pools/default\', error: path exist, request accepts it (id: [OwnerId: 72057594046644480, LocalPathId: 9], type: EPathTypeResourcePool, state: EPathStateNoChanges), source_location: ydb/core/tx/schemeshard/schemeshard__operation_create_resource_pool.cpp:92" severity: 1 }
2025-11-14T16:02:56.713807Z node 2 :KQP_COMPILE_ACTOR ERROR: kqp_compile_actor.cpp:570: Compilation failed, self: [2:7572612215353570834:2310], status: SCHEME_ERROR, issues: <main>: Error: Type annotation, code: 1030
    <main>:5:17: Error: At function: KiReadTable!
        <main>:5:17: Error: Cannot find table 'db.[/Root/PQ/Config/V2/Versions]' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions., code: 2003
2025-11-14T16:02:56.713943Z node 2 :KQP_SESSION WARN: kqp_session_actor.cpp:2517: SessionId: ydb://session/3?node_id=2&id=YjM2MDk4NjgtZTI5N2IwMy0yNzNlOWFmNy1jM2Y2NzVjYw==, ActorId: [2:7572612215353570720:2297], ActorState: ExecuteState, TraceId: 01ka1hp0f50dec6rktjn7245w2, ReplyQueryCompileError, status: SCHEME_ERROR, issues: { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 5 column: 17 } message: "At function: KiReadTable!" end_position { row: 5 column: 17 } severity: 1 issues { position { row: 5 column: 17 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Versions]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 5 column: 17 } issue_code: 2003 severity: 1 } } }, remove tx with tx_id: 
2025-11-14T16:02:56.714084Z node 2 :PERSQUEUE_CLUSTER_TRACKER ERROR: cluster_tracker.cpp:167: failed to list clusters: { Response { QueryIssues { message: "Type annotation" issue_code: 1030 severity: 1 issues { position { row: 5 column: 17 } message: "At function: KiReadTable!" end_position { row: 5 column: 17 } severity: 1 issues { position { row: 5 column: 17 } message: "Cannot find table \'db.[/Root/PQ/Config/V2/Versions]\' because it does not exist or you do not have access permissions. Please check correctness of table path and user permissions." end_position { row: 5 column: 17 } issue_code: 2003 severity: 1 } } } TxMeta { } } YdbStatus: SCHEME_ERROR ConsumedRu: 1 }
2025-11-14T16:02:56.728526Z node 1 :FLAT_TX_SCHEMESHARD WARN: schemeshard__operation.cpp:183: Operation part proposed ok, but propose itself is undo unsafe, suboperation type: ESchemeOpCreateTable, opId: 281474976720664:0, at schemeshard:  72057594046644480, first GetDB called at:  (GetDB first called at ydb/core/tx/schemeshard/schemeshard__operation_create_table.cpp:690)
=== Init DC: UPSERT INTO `/Root/PQ/Config/V2/Cluster` (name, balancer, local, enabled, weight) VALUES ("dc1", "localhost", true, true, 1000), ("dc2", "dc2.logbroker.yandex.net", false, true, 1000);
2025-11-14T16:02:56.779947Z node 1 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720665. Ctx: { TraceId: 01ka1hp0kf40ve3jz0wj6w9ys2, Database: , SessionId: ydb://session/3?node_id=1&id=OTNmOWM5NDItMTdiMjc0YjctNmViZDFiMTMtODY0ZDRlZjg=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
=== CheckClustersList. Subcribe to ClusterTracker from [1:7572612214870146772:3036] 
2025-11-14T16:02:57.186516Z node 1 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-11-14T16:02:57.188577Z node 2 :TX_CONVEYOR ERROR: log.cpp:841: fline=service.h:53;problem=unexpected event for task executor;ev_type=NActors::TEvents::TEvWakeup;
2025-11-14T16:03:01.183742Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[1:7572612214870145289:2088];send_to=[0:7307199536658146131:7762515];
2025-11-14T16:03:01.183781Z node 1 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
2025-11-14T16:03:01.190323Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=table_exists.cpp:59;actor=TTableExistsActor;event=timeout;self_id=[2:7572612215353570463:2154];send_to=[0:7307199536658146131:7762515];
2025-11-14T16:03:01.190423Z node 2 :METADATA_PROVIDER ERROR: log.cpp:841: fline=accessor_snapshot_base.cpp:71;action=cannot detect path existence;path=//Root/.metadata/initialization/migrations;error=timeout;
=== CheckClustersList. Ok
PQ Client: create topic: rt3.dc1--topic1 with 1 partitions
CallPersQueueGRPC request to localhost:2468
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--topic1"
  }
}
2025-11-14T16:03:01.984629Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 128
ErrorReason: "the following topics are not created: rt3.dc1--topic1, Marker# PQ95"
ErrorCode: UNKNOWN_TOPIC
CallPersQueueGRPC request to localhost:2468
MetaRequest {
  CmdCreateTopic {
    Topic: "rt3.dc1--topic1"
    NumPartitions: 1
    Config {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 6291456
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
  }
}
CallPersQueueGRPC response:
Status: 129
ProxyErrorCode: 53
SchemeStatus: 1
FlatTxId {
  TxId: 281474976720671
  SchemeShardTabletId: 72057594046644480
  PathId: 13
}
ErrorCode: OK
AddTopic: rt3.dc1--topic1
===Run query:``DECLARE $version as Int64; DECLARE $path AS Utf8; DECLARE $cluster as Utf8; UPSERT INTO `/Root/PQ/Config/V2/Topics` (path, dc) VALUES ($path, $cluster); UPSERT INTO `/Root/PQ/Config/V2/Versions` (name, version) VALUES ("Topics", $version);`` with topic = topic1, dc = dc1
2025-11-14T16:03:02.000703Z node 1 :PERSQUEUE_READ_BALANCER INFO: read_balancer__balancing.cpp:1655: [72075186224037893][] pipe [1:7572612236344983692:3278] connected; active server actors: 1
2025-11-14T16:03:02.000760Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer__balancing.cpp:1519: [72075186224037893][rt3.dc1--topic1] updating configuration. Deleted partitions []. Added partitions [0]
2025-11-14T16:03:02.001253Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:971: [72075186224037893][rt3.dc1--topic1] Discovered subdomain [OwnerId: 72057594046644480, LocalPathId: 1] state, outOfSpace = 0 at RB 72075186224037893
2025-11-14T16:03:02.001305Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:140: [72075186224037893][rt3.dc1--topic1] BALANCER INIT DONE for rt3.dc1--topic1: (0, 72075186224037892) 
2025-11-14T16:03:02.009814Z node 1 :PERSQUEUE_READ_BALANCER DEBUG: read_balancer.cpp:404: [72075186224037893][rt3.dc1--topic1] TEvClientConnected TabletId 72075186224037892, NodeId 2, Generation 1
2025-11-14T16:03:02.008655Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2960: [PQ: 72075186224037892] Handle TEvInterconnect::TEvNodeInfo
2025-11-14T16:03:02.009358Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2992: [PQ: 72075186224037892] Transactions request. From tx_00000000000000000000, To tx_18446744073709551615
2025-11-14T16:03:02.009426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:787: [PQ: 72075186224037892] no config, start with empty partitions and default config
2025-11-14T16:03:02.009434Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:913: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-11-14T16:03:02.009453Z node 2 :PERSQUEUE INFO: pq_impl.cpp:612: [PQ: 72075186224037892] doesn't have tx writes info
2025-11-14T16:03:02.009605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612236344983690:3277], now have 1 active actors on pipe
2025-11-14T16:03:02.009610Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951002:2452], now have 1 active actors on pipe
2025-11-14T16:03:02.009884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3570: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-14T16:03:02.010966Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-14T16:03:02.012794Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951035:3318], now have 1 active actors on pipe
2025-11-14T16:03:02.013091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3570: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-14T16:03:02.013964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-14T16:03:02.014222Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitConfigStep
2025-11-14T16:03:02.014243Z node 2 :PERSQUEUE DEBUG: partition.cpp:1316: [72075186224037892][Partition][0][StateInit] HandleOnInit TEvPQ::TEvProposePartitionConfig
2025-11-14T16:03:02.014317Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitInternalFieldsStep
2025-11-14T16:03:02.014460Z node 2 :PERSQUEUE INFO: partition_init.cpp:1107: [72075186224037892][Partition][0][StateInit] bootstrapping 0 [2:7572612241123374853:2364]
2025-11-14T16:03:02.015198Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:81: [rt3.dc1--topic1:0:Initializer] Start initializing step TInitFieldsStep
2025-11-14T16:03:02.015203Z node 2 :PERSQUEUE DEBUG: partition_init.cpp:61: [rt3.dc1--topic1:0:Initializer] Initializing completed.
2025-11-14T16:03:02.015209Z node 2 :PERSQUEUE INFO: partition.cpp:708: [72075186224037892][Partition][0][StateInit] init complete for topic 'rt3.dc1--topic1' partition 0 generation 1 [2:7572612241123374853:2364]
2025-11-14T16:03:02.015216Z node 2 :PERSQUEUE DEBUG: partition.cpp:728: [72075186224037892][Partition][0][StateInit] SYNC INIT topic rt3.dc1--topic1 partitition 0 so 0 endOffset 0 Head Offset 0 PartNo 0 PackedSize 0 count 0 nextOffset 0 batches 0
2025-11-14T16:03:02.015224Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:107: [72075186224037892][Partition][0][StateIdle] Initializing MLP Consumers: 0
2025-11-14T16:03:02.015528Z node 2 :PERSQUEUE DEBUG: partition.cpp:4406: [72075186224037892][Partition][0][StateIdle] Process pending events. Count 1
2025-11-14T16:03:02.015538Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.015556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2400: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TTransaction[ProposeConfig])
2025-11-14T16:03:02.015566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.015569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.015572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.015573Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.015595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.015598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.015600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.015602Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.015604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.015612Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:930: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-11-14T16:03:02.015619Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:70: [72075186224037892][Partition][0][StateIdle] Process MLP pending events. Count 0
2025-11-14T16:03:02.015866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3570: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-14T16:03:02.015907Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.018108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-14T16:03:02.018156Z node 2 :PERSQUEUE DEBUG: partition.cpp:1428: [72075186224037892][Partition][0][StateIdle] Handle TEvPQ::TEvTxCommit Step 1763136182059, TxId 281474976720671
2025-11-14T16:03:02.018170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.018174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.018177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.018183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2464: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TTransaction[ProposeConfig])
2025-11-14T16:03:02.018197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.018199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.018202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.018244Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.018965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.019005Z node 2 :PERSQUEUE DEBUG: partition_mlp.cpp:107: [72075186224037892][Partition][0][StateIdle] Initializing MLP Consumers: 0
2025-11-14T16:03:02.019037Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.019043Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.019046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.019048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.019050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.019052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.019058Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.019170Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:398: [PQ: 72075186224037892] Apply new config PartitionConfig { LifetimeSeconds: 86400 LowWatermark: 6291456 SourceIdLifetimeSeconds: 86400 WriteSpeedInBytesPerSecond: 20000000 BurstSize: 20000000 TotalPartitions: 1 SourceIdMaxCounts: 6000000 } PartitionIds: 0 TopicName: "rt3.dc1--topic1" Version: 0 LocalDC: true Codecs { Ids: 0 Ids: 1 Ids: 2 Codecs: "raw" Codecs: "gzip" Codecs: "lzop" } TopicPath: "/Root/PQ/rt3.dc1--topic1" YcCloudId: "" YcFolderId: "" YdbDatabaseId: "" YdbDatabasePath: "/Root" Partitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } AllPartitions { PartitionId: 0 Status: Active CreateVersion: 1 TabletId: 0 } Consumers { Name: "user" ReadFromTimestampsMs: 0 FormatVersion: 0 Version: 0 Generation: 0 } MonitoringProjectId: ""
2025-11-14T16:03:02.019188Z node 2 :PERSQUEUE NOTICE: pq_impl.cpp:913: [PQ: 72075186224037892] disable metering: reason# billing is not enabled in BillingMeteringConfig
2025-11-14T16:03:02.019230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3570: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-14T16:03:02.019607Z node 1 :KQP_EXECUTER ERROR: kqp_planner.cpp:123: TxId: 281474976720672. Ctx: { TraceId: 01ka1hp5qd1y8grzncqrz77ama, Database: , SessionId: ydb://session/3?node_id=1&id=MjI1YjU4ZTgtZTBjNjJkYWYtNWI1MDg1MGEtYTA1NjhhNjQ=, PoolId: default, DatabaseId: /Root}. Database not set, use /Root
2025-11-14T16:03:02.021367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
2025-11-14T16:03:02.021398Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:3570: [PQ: 72075186224037892] Send TEvKeyValue::TEvRequest (WRITE_TX_COOKIE)
2025-11-14T16:03:02.022074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:1045: [PQ: 72075186224037892] Handle TEvKeyValue::TEvResponse (WRITE_TX_COOKIE)
===Query complete
TClient::Ls request: /Root/PQ/rt3.dc1--topic1
TClient::Ls response: Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--topic1"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720671
    CreateStep: 1763136182059
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--topic1"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 6291456
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 600000...
(TRUNCATED)
GetTopicVersionFromPath:  record Status: 1
StatusCode: SUCCESS
SchemeStatus: 0
PathDescription {
  Self {
    Name: "rt3.dc1--topic1"
    PathId: 13
    SchemeshardId: 72057594046644480
    PathType: EPathTypePersQueueGroup
    CreateFinished: true
    CreateTxId: 281474976720671
    CreateStep: 1763136182059
    ParentPathId: 2
    PathState: EPathStateNoChanges
    Owner: "root@builtin"
    ACL: ""
    EffectiveACL: ""
    PathVersion: 2
    PathSubType: EPathSubTypeEmpty
    Version {
      GeneralVersion: 2
      ACLVersion: 0
      EffectiveACLVersion: 0
      UserAttrsVersion: 1
      ChildrenVersion: 0
      PQVersion: 1
    }
    ChildrenExist: false
  }
  PersQueueGroup {
    Name: "rt3.dc1--topic1"
    PathId: 13
    TotalGroupCount: 1
    PartitionPerTablet: 5
    PQTabletConfig {
      PartitionConfig {
        LifetimeSeconds: 86400
        LowWatermark: 6291456
        SourceIdLifetimeSeconds: 86400
        WriteSpeedInBytesPerSecond: 20000000
        BurstSize: 20000000
        SourceIdMaxCounts: 6000000
      }
      LocalDC: true
      Codecs {
        Ids: 0
        Ids: 1
        Ids: 2
        Codecs: "raw"
        Codecs: "gzip"
        Codecs: "lzop"
      }
      YdbDatabasePath: "/Root"
      Consumers {
        Name: "user"
        ReadFromTimestampsMs: 0
        FormatVersion: 0
        Version: 0
      }
    }
    Partitions {
      PartitionId: 0
      TabletId: 72075186224037892
      Status: Active
    }
    AlterVersion: 1
    BalancerTabletID: 72075186224037893
    NextPartitionId: 1
  }
  DomainDescription {
    SchemeShardId_Depricated: 72057594046644480
    PathId_Depricated: 1
    ProcessingParams {
      Version: 1
      PlanResolution: 7
      Coordinators: 72057594046316545
      TimeCastBucketsPerMediator: 2
      Mediators: 72057594046382081
    }
    DomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    PathsInside: 12
    PathsLimit: 10000
    ShardsInside: 6
    ShardsLimit: 200000
    ResourcesDomainKey {
      SchemeShard: 72057594046644480
      PathId: 1
    }
    DiskSpaceUsage {
      Tables {
        TotalSize: 0
        DataSize: 0
        IndexSize: 0
      }
      Topics {
        ReserveSize: 0
        AccountSize: 0
        DataSize: 0
        UsedReserveSize: 0
      }
    }
    PQPartitionsInside: 1
    PQPartitionsLimit: 1000000
    SchemeLimits {
      MaxDepth: 32
      MaxPaths: 10000
      MaxChildrenInDir: 100000
      MaxAclBytesSize: 10240
      MaxTableColumns: 200
      MaxTableColumnNameLength: 255
      MaxTableKeyColumns: 30
      MaxTableIndices: 20
      MaxShards: 200000
      MaxShardsInPath: 35000
      MaxConsistentCopyTargets: 10000
      MaxPathElementLength: 255
      ExtraPathSymbolsAllowed: "!\"#$%&\'()*+,-.:;<=>?@[\\]^_`{|}~"
      MaxPQPartitions: 1000000
      MaxTableCdcStreams: 5
      MaxExports: 10
      MaxImports: 10
      MaxColumnTableColumns: 10000
    }
  }
}
Path: "/Root/PQ/rt3.dc1--topic1"
 name rt3.dc1--topic1 version1
CallPersQueueGRPC request to localhost:2468
MetaRequest {
  CmdGetTopicMetadata {
    Topic: "rt3.dc1--topic1"
  }
}
2025-11-14T16:03:02.027681Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
MetaResponse {
  CmdGetTopicMetadataResult {
    TopicInfo {
      Topic: "rt3.dc1--topic1"
      NumPartitions: 1
      Config {
        PartitionConfig {
          LifetimeSeconds: 86400
          LowWatermark: 6291456
          SourceIdLifetimeSeconds: 86400
          WriteSpeedInBytesPerSecond: 20000000
          BurstSize: 20000000
          SourceIdMaxCounts: 6000000
        }
        Version: 1
        LocalDC: true
        Codecs {
          Ids: 0
          Ids: 1
          Ids: 2
          Codecs: "raw"
          Codecs: "gzip"
          Codecs: "lzop"
        }
        TopicPath: "/Root/PQ/rt3.dc1--topic1"
        YdbDatabasePath: "/Root"
        Consumers {
          Name: "user"
          ReadFromTimestampsMs: 0
          FormatVersion: 0
          Version: 0
        }
      }
      ErrorCode: OK
    }
  }
}
=== Topic created, have version: 1
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.028980Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.029432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951098:3359], now have 1 active actors on pipe
2025-11-14T16:03:02.029575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.029594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.029628Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|19c57bd8-b0309a3f-458ca7e9-1ac3ca06_0 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.029645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.029652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.029660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.029663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.029668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.029676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.029680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.029684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.029699Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.029718Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.029785Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|19c57bd8-b0309a3f-458ca7e9-1ac3ca06_0"
    Status: Active
    SeqNo: 0
  }
}
2025-11-14T16:03:02.030055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951098:3359] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|19c57bd8-b0309a3f-458ca7e9-1ac3ca06_0"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 0
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.036259Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.037052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951101:3360], now have 1 active actors on pipe
2025-11-14T16:03:02.037773Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.037791Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.037889Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 0 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.037905Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.037919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.037925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.037933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.037935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.037939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.037960Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 0
2025-11-14T16:03:02.038003Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 0 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 262214 count 1 nextOffset 1 batches 1
2025-11-14T16:03:02.038010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.038012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.038015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.038479Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 0,1 HeadOffset 0 endOffset 0 curOffset 1 d0000000000_00000000000000000000_00000_0000000001_00000? size 262204 WTime 1763136182038
2025-11-14T16:03:02.038516Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.038527Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 1 size 262204
2025-11-14T16:03:02.041259Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 0 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.041280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.041302Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.041309Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.041313Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.041316Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 0, partNo: 0, Offset: 0 is stored on disk
2025-11-14T16:03:02.041344Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:930: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 0 rrg 0
2025-11-14T16:03:02.041352Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:972: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 user user send read request for offset 0 initiated  queuesize 0 startOffset 0 ReadingTimestamp 1 rrg 0
2025-11-14T16:03:02.041355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.041357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.041358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.041360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.041361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.041367Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.041376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.041407Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 3 Topic 'rt3.dc1--topic1' partition 0 user user offset 0 partno 0 count 1 size 1024000 endOffset 1 max time lag 0ms effective offset 0
2025-11-14T16:03:02.041421Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 3 added 1 blobs, size 262204 count 1 last offset 0, current partition end offset: 1
2025-11-14T16:03:02.041428Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 3. Send blob request.
2025-11-14T16:03:02.041437Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.041450Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 3. All 1 blobs are from cache.
2025-11-14T16:03:02.041464Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 1 blobs
2025-11-14T16:03:02.041464Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.041582Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 1 count 1 size 262182 from pos 0 cbcount 1
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 0
    Offset: 0
    WriteTimestampMS: 1763136182037
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.041618Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1022: Topic 'rt3.dc1--topic1' partition 0 user user readTimeStamp done, result 1763136182037 queuesize 0 startOffset 0
2025-11-14T16:03:02.041679Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.041803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951101:3360] destroyed
2025-11-14T16:03:02.043312Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.043777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951105:3362], now have 1 active actors on pipe
2025-11-14T16:03:02.043930Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.043945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.043978Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f7e0a01a-865d7b3-76319d70-179a8f6_1 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.043992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.043996Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.044001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.044005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.044008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.044018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.044021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.044024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.044040Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.044054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.044111Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f7e0a01a-865d7b3-76319d70-179a8f6_1"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
2025-11-14T16:03:02.044315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951105:3362] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f7e0a01a-865d7b3-76319d70-179a8f6_1"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 1
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.047051Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.047543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951108:3363], now have 1 active actors on pipe
2025-11-14T16:03:02.047985Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.048004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.048024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 1 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.048036Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.048055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.048064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.048075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.048077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.048082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.048110Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 0
2025-11-14T16:03:02.048150Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 1 partNo 0 FormedBlobsCount 0 NewHead: Offset 1 PartNo 0 PackedSize 262214 count 1 nextOffset 2 batches 1
2025-11-14T16:03:02.048161Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.048164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.048167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.048293Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 1,1 HeadOffset 1 endOffset 1 curOffset 2 d0000000000_00000000000000000001_00000_0000000001_00000? size 262204 WTime 1763136182047
2025-11-14T16:03:02.048341Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.048357Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 1 partNo 0 count 1 size 262204
2025-11-14T16:03:02.050955Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 1 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.050986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.051006Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.051008Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.051017Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.051027Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 1, partNo: 0, Offset: 1 is stored on disk
2025-11-14T16:03:02.051067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.051076Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.051078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.051082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.051084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.051092Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.051103Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.051177Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.051486Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951108:3363] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 1
    Offset: 1
    WriteTimestampMS: 1763136182047
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.052177Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.052871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951112:3365], now have 1 active actors on pipe
2025-11-14T16:03:02.053033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.053047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.053084Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|415efbc6-cfadfc24-559960cb-acbc87c4_2 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.053098Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.053101Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.053107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.053108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.053112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.053122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.053128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.053130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.053144Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.053161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.053272Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.053501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951112:3365] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|415efbc6-cfadfc24-559960cb-acbc87c4_2"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|415efbc6-cfadfc24-559960cb-acbc87c4_2"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 2
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.055737Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.056152Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951115:3366], now have 1 active actors on pipe
2025-11-14T16:03:02.056546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.056566Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.056589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 2 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.056610Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.056625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.056629Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.056638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.056640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.056647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.056675Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 0
2025-11-14T16:03:02.056727Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 2 partNo 0 FormedBlobsCount 0 NewHead: Offset 2 PartNo 0 PackedSize 262214 count 1 nextOffset 3 batches 1
2025-11-14T16:03:02.056739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.056741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.056744Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.057210Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 2,1 HeadOffset 2 endOffset 2 curOffset 3 d0000000000_00000000000000000002_00000_0000000001_00000? size 262204 WTime 1763136182057
2025-11-14T16:03:02.057256Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.057267Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 2 partNo 0 count 1 size 262204
2025-11-14T16:03:02.060178Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 2 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.060207Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.060227Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.060230Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.060239Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.060250Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 2, partNo: 0, Offset: 2 is stored on disk
2025-11-14T16:03:02.060284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.060292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.060295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.060298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.060301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.060309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.060324Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.060444Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.060781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951115:3366] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 2
    Offset: 2
    WriteTimestampMS: 1763136182056
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.061528Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.061955Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951119:3368], now have 1 active actors on pipe
2025-11-14T16:03:02.062075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.062089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.062124Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|571d0975-4ea89ec-1a949918-68cc92c0_3 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.062133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.062136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.062141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.062144Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.062147Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.062157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.062159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.062164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.062176Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.062187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.062245Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.062483Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951119:3368] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|571d0975-4ea89ec-1a949918-68cc92c0_3"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|571d0975-4ea89ec-1a949918-68cc92c0_3"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 3
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.065380Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.066343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951122:3369], now have 1 active actors on pipe
2025-11-14T16:03:02.066772Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.066788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.066812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 3 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.066840Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.066859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.066866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.066876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.066882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.066886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.066912Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 0
2025-11-14T16:03:02.066946Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 3 partNo 0 FormedBlobsCount 0 NewHead: Offset 3 PartNo 0 PackedSize 262214 count 1 nextOffset 4 batches 1
2025-11-14T16:03:02.066954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.066956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.066959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.067282Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 3,1 HeadOffset 3 endOffset 3 curOffset 4 d0000000000_00000000000000000003_00000_0000000001_00000? size 262204 WTime 1763136182066
2025-11-14T16:03:02.067316Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.067328Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 3 partNo 0 count 1 size 262204
2025-11-14T16:03:02.069932Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 3 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.069960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.069980Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.069995Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.070001Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.070009Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 3, partNo: 0, Offset: 3 is stored on disk
2025-11-14T16:03:02.070049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.070057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.070059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.070063Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.070065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.070074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.070085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.070185Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 3
    Offset: 3
    WriteTimestampMS: 1763136182066
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.070797Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951122:3369] destroyed
2025-11-14T16:03:02.071321Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.071723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951126:3371], now have 1 active actors on pipe
2025-11-14T16:03:02.071786Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.071798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.071828Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|4db8b22e-c28d0ae6-7d610f7c-59590ad1_4 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.071845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.071848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.071852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.071854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.071858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.071867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.071869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.071872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.071884Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.071900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.071946Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.072167Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951126:3371] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|4db8b22e-c28d0ae6-7d610f7c-59590ad1_4"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|4db8b22e-c28d0ae6-7d610f7c-59590ad1_4"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 4
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.075319Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.075780Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951129:3372], now have 1 active actors on pipe
2025-11-14T16:03:02.076209Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.076221Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.076241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 4 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.076258Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.076272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.076276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.076286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.076292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.076297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.076328Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 0
2025-11-14T16:03:02.076365Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 4 partNo 0 FormedBlobsCount 0 NewHead: Offset 4 PartNo 0 PackedSize 262214 count 1 nextOffset 5 batches 1
2025-11-14T16:03:02.076377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.076380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.076382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.076855Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 4,1 HeadOffset 4 endOffset 4 curOffset 5 d0000000000_00000000000000000004_00000_0000000001_00000? size 262204 WTime 1763136182076
2025-11-14T16:03:02.076897Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.076933Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 4 partNo 0 count 1 size 262204
2025-11-14T16:03:02.080250Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 4 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.080272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.080290Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.080294Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.080298Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.080306Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 4, partNo: 0, Offset: 4 is stored on disk
2025-11-14T16:03:02.080332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.080340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.080342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.080345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.080347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.080356Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.080368Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.080468Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 4
    Offset: 4
    WriteTimestampMS: 1763136182076
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-14T16:03:02.080800Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951129:3372] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.081447Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.081842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951133:3374], now have 1 active actors on pipe
2025-11-14T16:03:02.081929Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.081944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.081976Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|75d07881-144ec7d7-196927a-dd94a2de_5 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.081990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.081998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.082003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.082005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.082009Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.082019Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.082023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.082026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.082039Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.082057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.082124Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|75d07881-144ec7d7-196927a-dd94a2de_5"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
2025-11-14T16:03:02.082603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951133:3374] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|75d07881-144ec7d7-196927a-dd94a2de_5"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 5
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.084639Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.085050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951136:3375], now have 1 active actors on pipe
2025-11-14T16:03:02.085488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.085504Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.085523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 5 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.085534Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.085548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.085556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.085567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.085574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.085578Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.085603Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 0
2025-11-14T16:03:02.085639Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 5 partNo 0 FormedBlobsCount 0 NewHead: Offset 5 PartNo 0 PackedSize 262214 count 1 nextOffset 6 batches 1
2025-11-14T16:03:02.085651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.085654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.085656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.085778Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 5,1 HeadOffset 5 endOffset 5 curOffset 6 d0000000000_00000000000000000005_00000_0000000001_00000? size 262204 WTime 1763136182085
2025-11-14T16:03:02.085821Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.085837Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 5 partNo 0 count 1 size 262204
2025-11-14T16:03:02.088188Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 5 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.088230Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.088245Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.088251Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.088261Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.088270Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 5, partNo: 0, Offset: 5 is stored on disk
2025-11-14T16:03:02.088309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.088317Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.088320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.088322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.088325Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.088333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.088344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.088462Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 5
    Offset: 5
    WriteTimestampMS: 1763136182085
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-14T16:03:02.088860Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951136:3375] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.089489Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.089939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951140:3377], now have 1 active actors on pipe
2025-11-14T16:03:02.090065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.090080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.090114Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|d9a307f8-3d132bcc-ab87489d-6da70bc9_6 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.090127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.090131Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.090136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.090139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.090143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.090154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.090157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.090160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.090172Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.090184Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.090244Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|d9a307f8-3d132bcc-ab87489d-6da70bc9_6"
    Status: Active
    SeqNo: 0
  }
}
2025-11-14T16:03:02.090535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951140:3377] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|d9a307f8-3d132bcc-ab87489d-6da70bc9_6"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 6
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.092693Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.093150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951143:3378], now have 1 active actors on pipe
2025-11-14T16:03:02.093533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.093548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.093569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 6 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.093585Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.093599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.093606Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.093615Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.093621Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.093625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.093654Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 0
2025-11-14T16:03:02.093692Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 6 partNo 0 FormedBlobsCount 0 NewHead: Offset 6 PartNo 0 PackedSize 262214 count 1 nextOffset 7 batches 1
2025-11-14T16:03:02.093703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.093705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.093707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.093839Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 6,1 HeadOffset 6 endOffset 6 curOffset 7 d0000000000_00000000000000000006_00000_0000000001_00000? size 262204 WTime 1763136182093
2025-11-14T16:03:02.093880Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.093896Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 6 partNo 0 count 1 size 262204
2025-11-14T16:03:02.096517Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 6 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.096545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.096563Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.096567Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.096578Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.096587Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 6, partNo: 0, Offset: 6 is stored on disk
2025-11-14T16:03:02.096628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.096638Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.096640Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.096642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.096644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.096652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.096662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.096725Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.097075Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951143:3378] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 6
    Offset: 6
    WriteTimestampMS: 1763136182093
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.097758Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.098199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951147:3380], now have 1 active actors on pipe
2025-11-14T16:03:02.098273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.098285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.098317Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b4c78bac-a6ea5159-d79d001a-1442a58c_7 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.098330Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.098334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.098338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.098341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.098345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.098354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.098357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.098361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.098371Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.098380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.098446Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b4c78bac-a6ea5159-d79d001a-1442a58c_7"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
2025-11-14T16:03:02.098727Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951147:3380] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b4c78bac-a6ea5159-d79d001a-1442a58c_7"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 7
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.100824Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.101271Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951150:3381], now have 1 active actors on pipe
2025-11-14T16:03:02.101788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.101803Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.101824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 7 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.101839Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.101857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.101865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.101875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.101882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.101886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.101915Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 0
2025-11-14T16:03:02.101967Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 7 partNo 0 FormedBlobsCount 0 NewHead: Offset 7 PartNo 0 PackedSize 262214 count 1 nextOffset 8 batches 1
2025-11-14T16:03:02.101983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.101986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.101989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.102121Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 7,1 HeadOffset 7 endOffset 7 curOffset 8 d0000000000_00000000000000000007_00000_0000000001_00000? size 262204 WTime 1763136182101
2025-11-14T16:03:02.102164Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.102183Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 7 partNo 0 count 1 size 262204
2025-11-14T16:03:02.104746Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 7 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.104774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.104792Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.104799Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.104807Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.104816Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 7, partNo: 0, Offset: 7 is stored on disk
2025-11-14T16:03:02.104854Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.104861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.104863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.104866Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.104868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.104877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.104887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.104964Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.105309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951150:3381] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 7
    Offset: 7
    WriteTimestampMS: 1763136182101
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.106097Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.106505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951154:3383], now have 1 active actors on pipe
2025-11-14T16:03:02.106590Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.106604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.106640Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|cc270c6b-42d95a71-f2b117f2-3d05601_8 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.106653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.106656Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.106661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.106664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.106667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.106677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.106680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.106684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.106697Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.106709Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.106795Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|cc270c6b-42d95a71-f2b117f2-3d05601_8"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
2025-11-14T16:03:02.107109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951154:3383] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|cc270c6b-42d95a71-f2b117f2-3d05601_8"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 8
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.109223Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.109575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951157:3384], now have 1 active actors on pipe
2025-11-14T16:03:02.109975Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.109993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.110016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 8 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.110026Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.110040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.110049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.110058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.110071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.110080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.110107Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 0
2025-11-14T16:03:02.110139Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 8 partNo 0 FormedBlobsCount 0 NewHead: Offset 8 PartNo 0 PackedSize 262214 count 1 nextOffset 9 batches 1
2025-11-14T16:03:02.110150Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.110153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.110156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.110279Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 8,1 HeadOffset 8 endOffset 8 curOffset 9 d0000000000_00000000000000000008_00000_0000000001_00000? size 262204 WTime 1763136182109
2025-11-14T16:03:02.110324Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.110341Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 8 partNo 0 count 1 size 262204
2025-11-14T16:03:02.112765Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 8 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.112791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.112809Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.112822Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.112822Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.112836Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 8, partNo: 0, Offset: 8 is stored on disk
2025-11-14T16:03:02.112867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.112875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.112877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.112880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.112882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.112888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.112916Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.113048Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.113359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951157:3384] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 8
    Offset: 8
    WriteTimestampMS: 1763136182109
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.114099Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.114692Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|4ef849cf-f9683534-e18396ac-2d34ba26_9"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|4ef849cf-f9683534-e18396ac-2d34ba26_9"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 9
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.114462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951161:3386], now have 1 active actors on pipe
2025-11-14T16:03:02.114531Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.114537Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.114567Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|4ef849cf-f9683534-e18396ac-2d34ba26_9 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.114574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.114577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.114582Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.114584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.114588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.114596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.114600Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.114603Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.114620Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.114635Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.114890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951161:3386] destroyed
2025-11-14T16:03:02.115520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.115523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.115525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.115528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.115530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.117507Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.117978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951164:3387], now have 1 active actors on pipe
2025-11-14T16:03:02.118402Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.118414Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.118434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 9 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.118448Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.118462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.118466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.118473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.118475Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.118479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.118505Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 0
2025-11-14T16:03:02.118535Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 9 partNo 0 FormedBlobsCount 0 NewHead: Offset 9 PartNo 0 PackedSize 262214 count 1 nextOffset 10 batches 1
2025-11-14T16:03:02.118542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.118544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.118548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.118658Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 9,1 HeadOffset 9 endOffset 9 curOffset 10 d0000000000_00000000000000000009_00000_0000000001_00000? size 262204 WTime 1763136182118
2025-11-14T16:03:02.118695Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.118715Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 9 partNo 0 count 1 size 262204
2025-11-14T16:03:02.121662Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 9 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.121692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.121714Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.121722Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.121720Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.121732Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 9, partNo: 0, Offset: 9 is stored on disk
2025-11-14T16:03:02.121765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.121770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.121773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.121775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.121777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.121784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.121795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.121884Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 9
    Offset: 9
    WriteTimestampMS: 1763136182118
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.122465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951164:3387] destroyed
2025-11-14T16:03:02.123023Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.123487Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951168:3389], now have 1 active actors on pipe
2025-11-14T16:03:02.133087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.133114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.133160Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a584c817-53288667-87334295-ac7e27d7_10 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.133185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.133189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.133195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.133198Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.133203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.133215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.133219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.133225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.133243Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.133265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.133350Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.133761Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951168:3389] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a584c817-53288667-87334295-ac7e27d7_10"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a584c817-53288667-87334295-ac7e27d7_10"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 10
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.135953Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.136386Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951174:3390], now have 1 active actors on pipe
2025-11-14T16:03:02.136806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.136821Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.136843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 10 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.136853Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.136869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.136877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.136887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.136889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.136893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.136940Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 0
2025-11-14T16:03:02.136975Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 10 partNo 0 FormedBlobsCount 0 NewHead: Offset 10 PartNo 0 PackedSize 262214 count 1 nextOffset 11 batches 1
2025-11-14T16:03:02.136985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.136988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.136993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.137117Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 10,1 HeadOffset 10 endOffset 10 curOffset 11 d0000000000_00000000000000000010_00000_0000000001_00000? size 262204 WTime 1763136182136
2025-11-14T16:03:02.137159Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.137186Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 10 partNo 0 count 1 size 262204
2025-11-14T16:03:02.139714Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 10 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.139741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.139758Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.139772Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.139782Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.139798Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 10, partNo: 0, Offset: 10 is stored on disk
2025-11-14T16:03:02.139832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.139840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.139843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.139846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.139848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.139857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.139868Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.139947Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.140260Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951174:3390] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 10
    Offset: 10
    WriteTimestampMS: 1763136182136
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.141037Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.145229Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951179:3393], now have 1 active actors on pipe
2025-11-14T16:03:02.145370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.145381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.145419Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ba69c0c5-f65a4750-461481d3-46c8452e_11 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.145430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.145434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.145439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.145442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.145446Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.145456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.145459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.145463Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.145480Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.145496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.145603Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.147808Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951179:3393] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ba69c0c5-f65a4750-461481d3-46c8452e_11"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ba69c0c5-f65a4750-461481d3-46c8452e_11"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 11
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.151146Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.152126Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951182:3394], now have 1 active actors on pipe
2025-11-14T16:03:02.152683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.152697Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.152720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 11 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.152734Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.152751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.152754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.152766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.152768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.152775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.152803Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 0
2025-11-14T16:03:02.152843Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 11 partNo 0 FormedBlobsCount 0 NewHead: Offset 11 PartNo 0 PackedSize 262214 count 1 nextOffset 12 batches 1
2025-11-14T16:03:02.152849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.152851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.152855Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.153051Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 11,1 HeadOffset 11 endOffset 11 curOffset 12 d0000000000_00000000000000000011_00000_0000000001_00000? size 262204 WTime 1763136182152
2025-11-14T16:03:02.153087Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.153104Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 11 partNo 0 count 1 size 262204
2025-11-14T16:03:02.155730Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 11 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.155757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.155777Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.155784Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.155792Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.155802Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 11, partNo: 0, Offset: 11 is stored on disk
2025-11-14T16:03:02.155835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.155843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.155845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.155847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.155849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.155856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.155866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.155978Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.156361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951182:3394] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 11
    Offset: 11
    WriteTimestampMS: 1763136182152
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.157376Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.157961Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951186:3396], now have 1 active actors on pipe
2025-11-14T16:03:02.158138Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.158150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.158182Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|48731dd0-768f5c36-4d4c40c2-d1e243a9_12 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.158196Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.158199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.158205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.158212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.158216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.158235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.158243Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.158246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.158260Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.158276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.158416Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|48731dd0-768f5c36-4d4c40c2-d1e243a9_12"
    Status: Active
    SeqNo: 0
  }
}
2025-11-14T16:03:02.158766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951186:3396] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|48731dd0-768f5c36-4d4c40c2-d1e243a9_12"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 12
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.163808Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.164507Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951189:3397], now have 1 active actors on pipe
2025-11-14T16:03:02.164890Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.164919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.164943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 12 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.164959Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.164986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.164993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.165003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.165010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.165016Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.165049Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 0
2025-11-14T16:03:02.165101Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 12 partNo 0 FormedBlobsCount 0 NewHead: Offset 12 PartNo 0 PackedSize 262214 count 1 nextOffset 13 batches 1
2025-11-14T16:03:02.165112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.165118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.165121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.165283Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 12,1 HeadOffset 12 endOffset 12 curOffset 13 d0000000000_00000000000000000012_00000_0000000001_00000? size 262204 WTime 1763136182165
2025-11-14T16:03:02.165329Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.165342Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 12 partNo 0 count 1 size 262204
2025-11-14T16:03:02.169876Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 12 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.169898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.169918Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.169926Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.169949Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 12, partNo: 0, Offset: 12 is stored on disk
2025-11-14T16:03:02.169976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.169982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.169984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.169986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.169988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.169995Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.170004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.170060Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.170534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951189:3397] destroyed
2025-11-14T16:03:02.170131Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 12
    Offset: 12
    WriteTimestampMS: 1763136182164
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.177070Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.177637Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951193:3399], now have 1 active actors on pipe
2025-11-14T16:03:02.177853Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.177878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.177928Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|32057839-8a423083-e1771bf4-9964e352_13 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.177943Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.177948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.177955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.177959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.177969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.177994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.177998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.178002Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.178024Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.178050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.178159Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.178525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951193:3399] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|32057839-8a423083-e1771bf4-9964e352_13"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|32057839-8a423083-e1771bf4-9964e352_13"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 13
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.181584Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.182043Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951197:3401], now have 1 active actors on pipe
2025-11-14T16:03:02.185918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.185943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.185974Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 13 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.185992Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.186011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.186015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.186028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.186031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.186036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.186071Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 0
2025-11-14T16:03:02.186113Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 13 partNo 0 FormedBlobsCount 0 NewHead: Offset 13 PartNo 0 PackedSize 262214 count 1 nextOffset 14 batches 1
2025-11-14T16:03:02.186121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.186125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.186129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.186277Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 13,1 HeadOffset 13 endOffset 13 curOffset 14 d0000000000_00000000000000000013_00000_0000000001_00000? size 262204 WTime 1763136182186
2025-11-14T16:03:02.186320Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.186333Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 13 partNo 0 count 1 size 262204
2025-11-14T16:03:02.188599Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 13 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.188643Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.188706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.188737Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.188749Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.188762Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 13, partNo: 0, Offset: 13 is stored on disk
2025-11-14T16:03:02.188803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.188807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.188809Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.188812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.188815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.188826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.188841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.188954Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 13
    Offset: 13
    WriteTimestampMS: 1763136182185
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-14T16:03:02.189418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951197:3401] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.190045Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.190439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951205:3405], now have 1 active actors on pipe
2025-11-14T16:03:02.190548Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.190569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.190611Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|efb8ee93-2b224c75-4309acfc-cb32eee3_14 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.190626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.190635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.190641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.190643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.190647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.190661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.190668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.190671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.190707Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.190728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.190835Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.191088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951205:3405] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|efb8ee93-2b224c75-4309acfc-cb32eee3_14"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|efb8ee93-2b224c75-4309acfc-cb32eee3_14"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 14
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.194452Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.194931Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951208:3406], now have 1 active actors on pipe
2025-11-14T16:03:02.195359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.195376Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.195403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 14 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.195417Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.195434Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.195439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.195450Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.195452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.195458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.195490Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 0
2025-11-14T16:03:02.195533Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 14 partNo 0 FormedBlobsCount 0 NewHead: Offset 14 PartNo 0 PackedSize 262214 count 1 nextOffset 15 batches 1
2025-11-14T16:03:02.195543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.195545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.195548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.195671Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 14,1 HeadOffset 14 endOffset 14 curOffset 15 d0000000000_00000000000000000014_00000_0000000001_00000? size 262204 WTime 1763136182195
2025-11-14T16:03:02.195714Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.195726Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 14 partNo 0 count 1 size 262204
2025-11-14T16:03:02.198669Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 14 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.198695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.198712Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.198731Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.198746Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.198761Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 14, partNo: 0, Offset: 14 is stored on disk
2025-11-14T16:03:02.198800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.198807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.198808Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.198810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.198812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.198820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.198830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.198920Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.199289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951208:3406] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 14
    Offset: 14
    WriteTimestampMS: 1763136182194
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.199889Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.200339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951212:3408], now have 1 active actors on pipe
2025-11-14T16:03:02.200426Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.200439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.200493Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f35d8b33-73345b0f-d5caca0-504bc331_15 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.200510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.200514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.200520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.200633Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.200522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.200537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.200548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.200551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.200554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.200568Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.200583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f35d8b33-73345b0f-d5caca0-504bc331_15"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
2025-11-14T16:03:02.200861Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951212:3408] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f35d8b33-73345b0f-d5caca0-504bc331_15"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 15
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.203331Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.203739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951215:3409], now have 1 active actors on pipe
2025-11-14T16:03:02.204155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.204172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.204196Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 15 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.204209Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.204228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.204237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.204248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.204255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.204259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.204290Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 0
2025-11-14T16:03:02.204361Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 15 partNo 0 FormedBlobsCount 0 NewHead: Offset 15 PartNo 0 PackedSize 262214 count 1 nextOffset 16 batches 1
2025-11-14T16:03:02.204374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.204378Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.204381Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.204509Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 15,1 HeadOffset 15 endOffset 15 curOffset 16 d0000000000_00000000000000000015_00000_0000000001_00000? size 262204 WTime 1763136182204
2025-11-14T16:03:02.204569Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.204587Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 15 partNo 0 count 1 size 262204
2025-11-14T16:03:02.206994Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 15 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.207022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.207045Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.207051Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.207056Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.207070Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 15, partNo: 0, Offset: 15 is stored on disk
2025-11-14T16:03:02.207117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.207120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.207122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.207125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.207127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.207136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.207149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.207232Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.207822Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951215:3409] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 15
    Offset: 15
    WriteTimestampMS: 1763136182204
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.208828Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.209224Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951219:3411], now have 1 active actors on pipe
2025-11-14T16:03:02.209413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.209428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.209465Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|2dcaee5c-27303c09-538f975-f2408c50_16 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.209482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.209490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.209512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.209519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.209524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.209537Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.209545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.209547Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.209564Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.209584Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.209732Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|2dcaee5c-27303c09-538f975-f2408c50_16"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
2025-11-14T16:03:02.210000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951219:3411] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|2dcaee5c-27303c09-538f975-f2408c50_16"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 16
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.212161Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.212639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951222:3412], now have 1 active actors on pipe
2025-11-14T16:03:02.212988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.213000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.213019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 16 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.213028Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.213042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.213049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.213058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.213061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.213065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.213091Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 0
2025-11-14T16:03:02.213127Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 16 partNo 0 FormedBlobsCount 0 NewHead: Offset 16 PartNo 0 PackedSize 262214 count 1 nextOffset 17 batches 1
2025-11-14T16:03:02.213135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.213138Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.213140Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.213466Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 16,1 HeadOffset 16 endOffset 16 curOffset 17 d0000000000_00000000000000000016_00000_0000000001_00000? size 262204 WTime 1763136182213
2025-11-14T16:03:02.213513Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.213531Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 16 partNo 0 count 1 size 262204
2025-11-14T16:03:02.215876Z node 2 :PERSQUEUE DEBUG: partition.cpp:2289: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-14T16:03:02.219734Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 16 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.219761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.219777Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.219786Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.219797Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.219808Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 16, partNo: 0, Offset: 16 is stored on disk
2025-11-14T16:03:02.219849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.219853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.219855Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.219859Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.219861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.219874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.219887Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.220001Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 16
    Offset: 16
    WriteTimestampMS: 1763136182212
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 6
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.220646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951222:3412] destroyed
2025-11-14T16:03:02.221037Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.221681Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951226:3414], now have 1 active actors on pipe
2025-11-14T16:03:02.221702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.221710Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.221750Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b1dbff56-b3932275-f45ec78f-ef5ea9ef_17 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.221766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.221773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.221781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.221786Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.221790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.221806Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.221812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.221815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.221832Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.221854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.221927Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b1dbff56-b3932275-f45ec78f-ef5ea9ef_17"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b1dbff56-b3932275-f45ec78f-ef5ea9ef_17"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 17
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.223062Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951226:3414] destroyed
2025-11-14T16:03:02.224517Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.225067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951229:3415], now have 1 active actors on pipe
2025-11-14T16:03:02.225505Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.225519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.225546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 17 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.225560Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.225576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.225580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.225592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.225595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.225599Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.225632Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 0
2025-11-14T16:03:02.225664Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 17 partNo 0 FormedBlobsCount 0 NewHead: Offset 17 PartNo 0 PackedSize 262214 count 1 nextOffset 18 batches 1
2025-11-14T16:03:02.225671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.225673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.225675Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.225762Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 17,1 HeadOffset 17 endOffset 17 curOffset 18 d0000000000_00000000000000000017_00000_0000000001_00000? size 262204 WTime 1763136182225
2025-11-14T16:03:02.225812Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.225826Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 17 partNo 0 count 1 size 262204
2025-11-14T16:03:02.228196Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 17 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.228218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.228240Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.228245Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.228252Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.228262Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 17, partNo: 0, Offset: 17 is stored on disk
2025-11-14T16:03:02.228295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.228297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.228298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.228300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.228301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.228309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.228318Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.228409Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 17
    Offset: 17
    WriteTimestampMS: 1763136182225
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.229088Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951229:3415] destroyed
2025-11-14T16:03:02.229495Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.229934Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951233:3417], now have 1 active actors on pipe
2025-11-14T16:03:02.230053Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.230064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.230099Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ad603ffd-30da282c-7dbf382-e2754a4b_18 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.230108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.230111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.230117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.230119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.230123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.230133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.230136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.230139Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.230153Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.230183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.230330Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ad603ffd-30da282c-7dbf382-e2754a4b_18"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
2025-11-14T16:03:02.230766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951233:3417] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ad603ffd-30da282c-7dbf382-e2754a4b_18"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 18
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.233368Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.233896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951236:3418], now have 1 active actors on pipe
2025-11-14T16:03:02.234319Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.234335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.234361Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 18 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.234380Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.234404Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.234415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.234428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.234432Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.234437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.234468Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 0
2025-11-14T16:03:02.234514Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 18 partNo 0 FormedBlobsCount 0 NewHead: Offset 18 PartNo 0 PackedSize 262214 count 1 nextOffset 19 batches 1
2025-11-14T16:03:02.234527Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.234530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.234533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.234657Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 18,1 HeadOffset 18 endOffset 18 curOffset 19 d0000000000_00000000000000000018_00000_0000000001_00000? size 262204 WTime 1763136182234
2025-11-14T16:03:02.234704Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.234722Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 18 partNo 0 count 1 size 262204
2025-11-14T16:03:02.238502Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 18 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.238531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.238555Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.238567Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.238579Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 18, partNo: 0, Offset: 18 is stored on disk
2025-11-14T16:03:02.238622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.238626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.238629Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.238632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.238636Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.238648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.238661Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.238709Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.238789Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.239203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951236:3418] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 18
    Offset: 18
    WriteTimestampMS: 1763136182234
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.241371Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.241820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951240:3420], now have 1 active actors on pipe
2025-11-14T16:03:02.241926Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.241939Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.241974Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f76ad204-7ff04eb7-49e009e3-1f479ef4_19 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.241989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.241992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.241998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.242001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.242004Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.242015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.242022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.242026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.242041Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.242059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.242119Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f76ad204-7ff04eb7-49e009e3-1f479ef4_19"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
2025-11-14T16:03:02.242491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951240:3420] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f76ad204-7ff04eb7-49e009e3-1f479ef4_19"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 19
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.244773Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.245253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951243:3421], now have 1 active actors on pipe
2025-11-14T16:03:02.245643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.245657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.245680Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 19 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.245695Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.245717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.245725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.245737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.245743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.245747Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.245780Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 0
2025-11-14T16:03:02.245822Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 19 partNo 0 FormedBlobsCount 0 NewHead: Offset 19 PartNo 0 PackedSize 262214 count 1 nextOffset 20 batches 1
2025-11-14T16:03:02.245834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.245841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.245844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.245940Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 19,1 HeadOffset 19 endOffset 19 curOffset 20 d0000000000_00000000000000000019_00000_0000000001_00000? size 262204 WTime 1763136182244
2025-11-14T16:03:02.245975Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.245989Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 19 partNo 0 count 1 size 262204
2025-11-14T16:03:02.248275Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 19 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.248300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.248330Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.249035Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:490: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-11-14T16:03:02.249087Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.249119Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.249139Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 19, partNo: 0, Offset: 19 is stored on disk
2025-11-14T16:03:02.249205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.249221Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.249223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.249226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.249237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.249252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.249272Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.249384Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.249728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951243:3421] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 19
    Offset: 19
    WriteTimestampMS: 1763136182244
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.253996Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951247:3423], now have 1 active actors on pipe
2025-11-14T16:03:02.253529Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.254250Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.254264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.254309Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|bb67b0b3-a4cd7dc7-e7837809-ae3272c6_20 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.254323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.254328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.254335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.254337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.254342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.254353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.254357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.254361Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.254381Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.254399Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.254605Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.254938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951247:3423] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|bb67b0b3-a4cd7dc7-e7837809-ae3272c6_20"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|bb67b0b3-a4cd7dc7-e7837809-ae3272c6_20"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 20
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.257669Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.258175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951250:3424], now have 1 active actors on pipe
2025-11-14T16:03:02.258604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.258622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.258646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 20 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.258665Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.258679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.258692Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.258710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.258712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.258717Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.258747Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 0
2025-11-14T16:03:02.258850Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 20 partNo 0 FormedBlobsCount 0 NewHead: Offset 20 PartNo 0 PackedSize 262214 count 1 nextOffset 21 batches 1
2025-11-14T16:03:02.258858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.258860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.258863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.259001Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 20,1 HeadOffset 20 endOffset 20 curOffset 21 d0000000000_00000000000000000020_00000_0000000001_00000? size 262204 WTime 1763136182257
2025-11-14T16:03:02.259040Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.259051Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 20 partNo 0 count 1 size 262204
2025-11-14T16:03:02.261800Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 20 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.261836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.261856Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.261860Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.261867Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.261876Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 20, partNo: 0, Offset: 20 is stored on disk
2025-11-14T16:03:02.261920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.261931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.261933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.261938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.261940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.261946Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.261956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.262085Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 20
    Offset: 20
    WriteTimestampMS: 1763136182257
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-14T16:03:02.262495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951250:3424] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.263117Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.263522Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951254:3426], now have 1 active actors on pipe
2025-11-14T16:03:02.263623Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.263648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.263686Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|50ec7040-d35fc7f-259deb7f-2f7fe23b_21 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.263702Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.263705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.263712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.263714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.263719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.263729Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.263736Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.263740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.263758Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.263778Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.263871Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|50ec7040-d35fc7f-259deb7f-2f7fe23b_21"
    Status: Active
    SeqNo: 0
  }
}
2025-11-14T16:03:02.264150Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951254:3426] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|50ec7040-d35fc7f-259deb7f-2f7fe23b_21"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 21
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.266433Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.267121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951257:3427], now have 1 active actors on pipe
2025-11-14T16:03:02.267599Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.267617Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.267644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 21 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.267666Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.267683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.267687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.267700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.267703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.267708Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.267756Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 0
2025-11-14T16:03:02.267798Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 21 partNo 0 FormedBlobsCount 0 NewHead: Offset 21 PartNo 0 PackedSize 262214 count 1 nextOffset 22 batches 1
2025-11-14T16:03:02.267807Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.267810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.267814Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.267942Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 21,1 HeadOffset 21 endOffset 21 curOffset 22 d0000000000_00000000000000000021_00000_0000000001_00000? size 262204 WTime 1763136182267
2025-11-14T16:03:02.268001Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.268014Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 21 partNo 0 count 1 size 262204
2025-11-14T16:03:02.270963Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 21 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.270991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.271088Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.271106Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.271118Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 21, partNo: 0, Offset: 21 is stored on disk
2025-11-14T16:03:02.271168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.271176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.271179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.271181Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.271184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.271195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.271210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.271252Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.271326Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 21
    Offset: 21
    WriteTimestampMS: 1763136182266
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 1
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
2025-11-14T16:03:02.271774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951257:3427] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.272361Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.272765Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951261:3429], now have 1 active actors on pipe
2025-11-14T16:03:02.272851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.272878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.272938Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ffc7b32d-24c4d1f9-c85f03c2-4e19fb8f_22 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.272949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.272953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.272959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.272962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.273075Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.272966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.272990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.272994Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.272997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.273012Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.273027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ffc7b32d-24c4d1f9-c85f03c2-4e19fb8f_22"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
2025-11-14T16:03:02.273294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951261:3429] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ffc7b32d-24c4d1f9-c85f03c2-4e19fb8f_22"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 22
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.275576Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.275971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951264:3430], now have 1 active actors on pipe
2025-11-14T16:03:02.276380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.276397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.276422Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 22 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.276438Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.276456Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.276466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.276480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.276486Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.276490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.276518Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 0
2025-11-14T16:03:02.276573Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 22 partNo 0 FormedBlobsCount 0 NewHead: Offset 22 PartNo 0 PackedSize 262214 count 1 nextOffset 23 batches 1
2025-11-14T16:03:02.276590Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.276592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.276595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.276711Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 22,1 HeadOffset 22 endOffset 22 curOffset 23 d0000000000_00000000000000000022_00000_0000000001_00000? size 262204 WTime 1763136182276
2025-11-14T16:03:02.276755Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.276772Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 22 partNo 0 count 1 size 262204
2025-11-14T16:03:02.279053Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 22 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.279074Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.279092Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.279095Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.279105Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.279116Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 22, partNo: 0, Offset: 22 is stored on disk
2025-11-14T16:03:02.279159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.279166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.279169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.279171Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.279172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.279180Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.279193Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.279309Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.279664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951264:3430] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 22
    Offset: 22
    WriteTimestampMS: 1763136182276
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.280351Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.280788Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951268:3432], now have 1 active actors on pipe
2025-11-14T16:03:02.280944Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.280957Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.280997Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|d7c9acec-b9d223b2-b8d2df23-acee33e0_23 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.281013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.281017Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.281022Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.281024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.281028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.281038Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.281045Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.281049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.281062Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.281079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.281158Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|d7c9acec-b9d223b2-b8d2df23-acee33e0_23"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
2025-11-14T16:03:02.281494Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951268:3432] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|d7c9acec-b9d223b2-b8d2df23-acee33e0_23"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 23
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.283815Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.284312Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951271:3433], now have 1 active actors on pipe
2025-11-14T16:03:02.284742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.284760Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.284787Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 23 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.284798Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.284815Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.284819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.284831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.284834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.284839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.284866Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 0
2025-11-14T16:03:02.284917Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 23 partNo 0 FormedBlobsCount 0 NewHead: Offset 23 PartNo 0 PackedSize 262214 count 1 nextOffset 24 batches 1
2025-11-14T16:03:02.284926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.284929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.284935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.285063Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 23,1 HeadOffset 23 endOffset 23 curOffset 24 d0000000000_00000000000000000023_00000_0000000001_00000? size 262204 WTime 1763136182284
2025-11-14T16:03:02.285100Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.285108Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 23 partNo 0 count 1 size 262204
2025-11-14T16:03:02.287909Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 23 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.287933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.287960Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.287972Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.287982Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 23, partNo: 0, Offset: 23 is stored on disk
2025-11-14T16:03:02.288021Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.288023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.288024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.288026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.288028Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.288036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.288047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.288092Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.288175Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.288501Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951271:3433] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 23
    Offset: 23
    WriteTimestampMS: 1763136182284
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.289337Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.289732Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951275:3435], now have 1 active actors on pipe
2025-11-14T16:03:02.289804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.289820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.289863Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|96e6054b-10b6edcf-48c2b973-26d8c1ce_24 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.289879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.289883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.289889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.289892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.289896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.289906Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.289910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.289913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.289930Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.289946Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.289998Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.290204Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951275:3435] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|96e6054b-10b6edcf-48c2b973-26d8c1ce_24"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|96e6054b-10b6edcf-48c2b973-26d8c1ce_24"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 24
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.292564Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.293054Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951278:3436], now have 1 active actors on pipe
2025-11-14T16:03:02.293485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.293500Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.293533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 24 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.293545Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.293566Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.293574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.293585Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.293587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.293594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.293626Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 0
2025-11-14T16:03:02.293663Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 24 partNo 0 FormedBlobsCount 0 NewHead: Offset 24 PartNo 0 PackedSize 262214 count 1 nextOffset 25 batches 1
2025-11-14T16:03:02.293674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.293677Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.293679Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.293792Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 24,1 HeadOffset 24 endOffset 24 curOffset 25 d0000000000_00000000000000000024_00000_0000000001_00000? size 262204 WTime 1763136182293
2025-11-14T16:03:02.293832Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.293848Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 24 partNo 0 count 1 size 262204
2025-11-14T16:03:02.296189Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 24 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.296216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.296242Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.296252Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.296261Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 24, partNo: 0, Offset: 24 is stored on disk
2025-11-14T16:03:02.296294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.296298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.296300Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.296303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.296305Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.296316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.296330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.296370Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.296455Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 24
    Offset: 24
    WriteTimestampMS: 1763136182293
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-14T16:03:02.296771Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951278:3436] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.297442Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.303217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951282:3438], now have 1 active actors on pipe
2025-11-14T16:03:02.303336Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.303355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.303404Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|33fd7290-f427e7c1-c4f3b065-dfac8a7c_25 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.303422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.303430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.303437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.303444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.303448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.303462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.303470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.303473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.303493Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.303516Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.303600Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.304056Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951282:3438] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|33fd7290-f427e7c1-c4f3b065-dfac8a7c_25"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|33fd7290-f427e7c1-c4f3b065-dfac8a7c_25"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 25
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.307354Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.308190Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951289:3440], now have 1 active actors on pipe
2025-11-14T16:03:02.308359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.308373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.308397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 25 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.308419Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.308437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.308441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.308452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.308455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.308459Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.308489Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 0
2025-11-14T16:03:02.308528Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 25 partNo 0 FormedBlobsCount 0 NewHead: Offset 25 PartNo 0 PackedSize 262214 count 1 nextOffset 26 batches 1
2025-11-14T16:03:02.308539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.308544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.308546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.308661Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 25,1 HeadOffset 25 endOffset 25 curOffset 26 d0000000000_00000000000000000025_00000_0000000001_00000? size 262204 WTime 1763136182308
2025-11-14T16:03:02.308696Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.308709Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 25 partNo 0 count 1 size 262204
2025-11-14T16:03:02.311832Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 25 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.311862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.311896Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.312061Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.311913Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.311929Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 25, partNo: 0, Offset: 25 is stored on disk
2025-11-14T16:03:02.311966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.311968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.311969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.311971Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.311972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.311979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.311991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.312000Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 0 suffix '63' size 262204
CallPersQueueGRPC response:
2025-11-14T16:03:02.312442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951289:3440] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 25
    Offset: 25
    WriteTimestampMS: 1763136182307
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.313215Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.313682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951293:3442], now have 1 active actors on pipe
2025-11-14T16:03:02.313764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.313777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.313823Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|1ef14870-c8a8578-790d34b6-e6c2d9ab_26 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.313840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.313845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.313853Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.313856Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.313860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.313871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.313874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.313877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.313894Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.313909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.313991Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|1ef14870-c8a8578-790d34b6-e6c2d9ab_26"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
2025-11-14T16:03:02.314220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951293:3442] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|1ef14870-c8a8578-790d34b6-e6c2d9ab_26"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 26
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.316471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.316494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.316496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.316501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.316503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.316689Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.317381Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951296:3443], now have 1 active actors on pipe
2025-11-14T16:03:02.317862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.317880Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.317907Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 26 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.317930Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.317952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.317960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.317976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.317983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.317987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.318020Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 0
2025-11-14T16:03:02.318066Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 26 partNo 0 FormedBlobsCount 0 NewHead: Offset 26 PartNo 0 PackedSize 262214 count 1 nextOffset 27 batches 1
2025-11-14T16:03:02.318078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.318081Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.318084Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.318208Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 26,1 HeadOffset 26 endOffset 26 curOffset 27 d0000000000_00000000000000000026_00000_0000000001_00000? size 262204 WTime 1763136182317
2025-11-14T16:03:02.318258Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.318275Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 26 partNo 0 count 1 size 262204
2025-11-14T16:03:02.320870Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 26 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.320897Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.320936Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.320946Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.320957Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 26, partNo: 0, Offset: 26 is stored on disk
2025-11-14T16:03:02.321014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.321023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.321025Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.321029Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.321031Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.321041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.321055Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.321122Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.321222Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 26
    Offset: 26
    WriteTimestampMS: 1763136182317
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-14T16:03:02.321626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951296:3443] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.322287Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.322687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951300:3445], now have 1 active actors on pipe
2025-11-14T16:03:02.322812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.322825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.322863Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|746dd99-56422f33-6863833e-30424199_27 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.322880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.322892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.322898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.322900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.322903Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.322913Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.322916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.322919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.322935Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.322950Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.323048Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|746dd99-56422f33-6863833e-30424199_27"
    Status: Active
    SeqNo: 0
  }
}
2025-11-14T16:03:02.323334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951300:3445] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|746dd99-56422f33-6863833e-30424199_27"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 27
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.326047Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.326530Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951303:3446], now have 1 active actors on pipe
2025-11-14T16:03:02.326958Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.326972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.326998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 27 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.327015Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.327041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.327049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.327061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.327068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.327073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.327103Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 0
2025-11-14T16:03:02.327157Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 27 partNo 0 FormedBlobsCount 0 NewHead: Offset 27 PartNo 0 PackedSize 262214 count 1 nextOffset 28 batches 1
2025-11-14T16:03:02.327164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.327167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.327170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.327307Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 27,1 HeadOffset 27 endOffset 27 curOffset 28 d0000000000_00000000000000000027_00000_0000000001_00000? size 262204 WTime 1763136182326
2025-11-14T16:03:02.327351Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.327368Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 27 partNo 0 count 1 size 262204
2025-11-14T16:03:02.329535Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 27 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.329558Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.329591Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.329605Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.329618Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 27, partNo: 0, Offset: 27 is stored on disk
2025-11-14T16:03:02.329654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.329661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.329664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.329666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.329669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.329679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.329694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.329760Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.329834Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 27
    Offset: 27
    WriteTimestampMS: 1763136182326
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 1
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.330496Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951303:3446] destroyed
2025-11-14T16:03:02.330812Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.331203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951307:3448], now have 1 active actors on pipe
2025-11-14T16:03:02.331374Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.331382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.331416Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|5b7798d8-d0afb811-e14089b2-ec8b0348_28 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.331433Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.331437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.331442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.331445Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.331448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.331458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.331461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.331464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.331477Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.331489Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.331712Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.331967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951307:3448] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|5b7798d8-d0afb811-e14089b2-ec8b0348_28"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|5b7798d8-d0afb811-e14089b2-ec8b0348_28"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 28
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.334409Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.334833Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951310:3449], now have 1 active actors on pipe
2025-11-14T16:03:02.335306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.335317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.335343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 28 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.335355Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.335370Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.335374Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.335386Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.335388Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.335395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.335428Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 0
2025-11-14T16:03:02.335462Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 28 partNo 0 FormedBlobsCount 0 NewHead: Offset 28 PartNo 0 PackedSize 262214 count 1 nextOffset 29 batches 1
2025-11-14T16:03:02.335471Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.335473Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.335476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.335606Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 28,1 HeadOffset 28 endOffset 28 curOffset 29 d0000000000_00000000000000000028_00000_0000000001_00000? size 262204 WTime 1763136182335
2025-11-14T16:03:02.335643Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.335656Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 28 partNo 0 count 1 size 262204
2025-11-14T16:03:02.339186Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 28 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.339216Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.339233Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.339240Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.339251Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.339262Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 28, partNo: 0, Offset: 28 is stored on disk
2025-11-14T16:03:02.339302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.339311Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.339313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.339316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.339318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.339329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.339347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.339532Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 28
    Offset: 28
    WriteTimestampMS: 1763136182335
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.340026Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951310:3449] destroyed
2025-11-14T16:03:02.340665Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.341154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951314:3451], now have 1 active actors on pipe
2025-11-14T16:03:02.341421Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.341432Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.341467Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b9080c70-4730df3d-7a17cd47-5a77ebb6_29 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.341489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.341493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.341500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.341502Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.341508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.341519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.341522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.341525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.341542Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.341557Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.341777Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b9080c70-4730df3d-7a17cd47-5a77ebb6_29"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
2025-11-14T16:03:02.342353Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951314:3451] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b9080c70-4730df3d-7a17cd47-5a77ebb6_29"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 29
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.344489Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.344971Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951317:3452], now have 1 active actors on pipe
2025-11-14T16:03:02.345523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.345533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.345554Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 29 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.345563Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.345589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.345593Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.345605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.345608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.345612Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.345646Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 0
2025-11-14T16:03:02.345679Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 29 partNo 0 FormedBlobsCount 0 NewHead: Offset 29 PartNo 0 PackedSize 262214 count 1 nextOffset 30 batches 1
2025-11-14T16:03:02.345687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.345700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.345704Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.345826Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 29,1 HeadOffset 29 endOffset 29 curOffset 30 d0000000000_00000000000000000029_00000_0000000001_00000? size 262204 WTime 1763136182345
2025-11-14T16:03:02.345866Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.345891Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 29 partNo 0 count 1 size 262204
2025-11-14T16:03:02.349214Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 29
    Offset: 29
    WriteTimestampMS: 1763136182345
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.350585Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.351397Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|2cae2004-c5e2ffa6-ec72ae05-8b20aff5_30"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
2025-11-14T16:03:02.348864Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 29 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.348892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.348938Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.348950Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.348962Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 29, partNo: 0, Offset: 29 is stored on disk
2025-11-14T16:03:02.349001Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.349005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.349007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.349010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.349012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.349023Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.349036Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.349101Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.349577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951317:3452] destroyed
2025-11-14T16:03:02.351057Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951322:3455], now have 1 active actors on pipe
2025-11-14T16:03:02.351146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.351153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.351188Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|2cae2004-c5e2ffa6-ec72ae05-8b20aff5_30 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.351199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.351202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.351208Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|2cae2004-c5e2ffa6-ec72ae05-8b20aff5_30"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 30
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.351211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.351215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.351226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.351229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.351236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.351262Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.351275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.351698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951322:3455] destroyed
2025-11-14T16:03:02.354229Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.356161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951325:3456], now have 1 active actors on pipe
2025-11-14T16:03:02.356670Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.356688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.356711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 30 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.356727Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.356741Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.356745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.356756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.356758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.356764Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.356793Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 0
2025-11-14T16:03:02.356840Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 30 partNo 0 FormedBlobsCount 0 NewHead: Offset 30 PartNo 0 PackedSize 262214 count 1 nextOffset 31 batches 1
2025-11-14T16:03:02.356850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.356857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.356860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.357001Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 30,1 HeadOffset 30 endOffset 30 curOffset 31 d0000000000_00000000000000000030_00000_0000000001_00000? size 262204 WTime 1763136182356
2025-11-14T16:03:02.357046Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.357057Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 30 partNo 0 count 1 size 262204
2025-11-14T16:03:02.369897Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 30 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.369949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.369964Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.369976Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.369990Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.370019Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 30, partNo: 0, Offset: 30 is stored on disk
2025-11-14T16:03:02.370079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.370088Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.370090Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.370093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.370095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.370106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.370121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.370227Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 30
    Offset: 30
    WriteTimestampMS: 1763136182356
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 13
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.370572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951325:3456] destroyed
2025-11-14T16:03:02.371529Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.372244Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951329:3458], now have 1 active actors on pipe
2025-11-14T16:03:02.372269Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.372275Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.372311Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|89c0670f-ca6d51ba-3dc14bbd-51723a42_31 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.372324Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.372328Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.372334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.372336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.372340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.372350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.372353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.372358Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.372373Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.372388Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.372816Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951329:3458] destroyed
CallPersQueueGRPC response:
2025-11-14T16:03:02.372507Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|89c0670f-ca6d51ba-3dc14bbd-51723a42_31"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|89c0670f-ca6d51ba-3dc14bbd-51723a42_31"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 31
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.378816Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.380091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951332:3459], now have 1 active actors on pipe
2025-11-14T16:03:02.380325Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.380339Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.380359Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 31 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.380375Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.380408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.380416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.380427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.380429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.380436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.380467Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 0
2025-11-14T16:03:02.380522Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 31 partNo 0 FormedBlobsCount 0 NewHead: Offset 31 PartNo 0 PackedSize 262214 count 1 nextOffset 32 batches 1
2025-11-14T16:03:02.380536Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.380539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.380542Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.380672Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 31,1 HeadOffset 31 endOffset 31 curOffset 32 d0000000000_00000000000000000031_00000_0000000001_00000? size 262204 WTime 1763136182380
2025-11-14T16:03:02.380728Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.380741Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 31 partNo 0 count 1 size 262204
2025-11-14T16:03:02.383093Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 31 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.383125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.383146Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.383147Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.383161Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.383172Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 31, partNo: 0, Offset: 31 is stored on disk
2025-11-14T16:03:02.383210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.383218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.383223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.383226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.383228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.383238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000000_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000001_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000002_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000003_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383248Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000004_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000005_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000006_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000007_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000008_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383265Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000009_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000010_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000011_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000012_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383276Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000013_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383279Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000014_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383282Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000015_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383285Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000016_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000017_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000018_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000019_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000020_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000021_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000022_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000023_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000024_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383310Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000025_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383313Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000026_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383316Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000027_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383318Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000028_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383323Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000029_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383327Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000030_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000031_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.383332Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 32 keys were taken away. Let's read 8390528 bytes
2025-11-14T16:03:02.383346Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.383363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 32 blobs
2025-11-14T16:03:02.383366Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000000_00000_0000000001_00000?
2025-11-14T16:03:02.383376Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000001_00000_0000000001_00000?
2025-11-14T16:03:02.383387Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000002_00000_0000000001_00000?
2025-11-14T16:03:02.383402Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000003_00000_0000000001_00000?
2025-11-14T16:03:02.383410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000004_00000_0000000001_00000?
2025-11-14T16:03:02.383436Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000005_00000_0000000001_00000?
2025-11-14T16:03:02.383448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000006_00000_0000000001_00000?
2025-11-14T16:03:02.383455Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000007_00000_0000000001_00000?
2025-11-14T16:03:02.383463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000008_00000_0000000001_00000?
2025-11-14T16:03:02.383507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000009_00000_0000000001_00000?
2025-11-14T16:03:02.383520Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000010_00000_0000000001_00000?
2025-11-14T16:03:02.383530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000011_00000_0000000001_00000?
2025-11-14T16:03:02.383542Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000012_00000_0000000001_00000?
2025-11-14T16:03:02.383553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000013_00000_0000000001_00000?
2025-11-14T16:03:02.383565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000014_00000_0000000001_00000?
2025-11-14T16:03:02.383592Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000015_00000_0000000001_00000?
2025-11-14T16:03:02.383607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000016_00000_0000000001_00000?
2025-11-14T16:03:02.383674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000017_00000_0000000001_00000?
2025-11-14T16:03:02.383688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000018_00000_0000000001_00000?
2025-11-14T16:03:02.383695Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000019_00000_0000000001_00000?
2025-11-14T16:03:02.383703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000020_00000_0000000001_00000?
2025-11-14T16:03:02.383715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000021_00000_0000000001_00000?
2025-11-14T16:03:02.383424Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 31
    Offset: 31
    WriteTimestampMS: 1763136182380
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.383974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000022_00000_0000000001_00000?
2025-11-14T16:03:02.384343Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000023_00000_0000000001_00000?
2025-11-14T16:03:02.384360Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000024_00000_0000000001_00000?
2025-11-14T16:03:02.384368Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000025_00000_0000000001_00000?
2025-11-14T16:03:02.384378Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000026_00000_0000000001_00000?
2025-11-14T16:03:02.384391Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000027_00000_0000000001_00000?
2025-11-14T16:03:02.384401Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000028_00000_0000000001_00000?
2025-11-14T16:03:02.384413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000029_00000_0000000001_00000?
2025-11-14T16:03:02.384426Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000030_00000_0000000001_00000?
2025-11-14T16:03:02.384438Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000031_00000_0000000001_00000?
2025-11-14T16:03:02.384452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 32 blobs for compaction
2025-11-14T16:03:02.384465Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951332:3459] destroyed
2025-11-14T16:03:02.384478Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 1 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384481Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 1 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384484Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 2 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384488Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 3 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384492Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 4 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384495Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 5 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384498Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 6 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384502Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 7 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384506Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 8 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 9 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 10 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384520Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 11 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384525Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 12 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384527Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 13 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384530Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 14 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384533Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 15 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384536Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 16 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384544Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 17 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384547Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 18 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384550Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 19 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384554Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 20 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384557Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 21 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384560Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 22 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384563Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 23 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384567Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 24 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384571Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 25 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384574Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 26 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384576Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 27 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384579Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 28 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384582Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 29 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384585Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 30 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384587Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 31 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.384600Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 32 blobs are from cache.
2025-11-14T16:03:02.384616Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384622Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384625Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384627Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384630Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384634Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384637Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384640Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384643Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384646Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-14T16:03:02.384649Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384652Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384654Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/32] d0000000000_00000000000000000000_00000_0000000001_00000?
2025-11-14T16:03:02.384655Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.384658Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000000_00000_0000000001_00000?, parameters.CurOffset=0
2025-11-14T16:03:02.384660Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384664Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384667Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384670Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384673Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384677Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384679Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384682Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384685Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384688Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384691Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384694Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384697Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384700Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384704Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384706Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384709Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384711Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.384744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 0.0/1
2025-11-14T16:03:02.384751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 0 partNo 0
2025-11-14T16:03:02.384768Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 0 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 262214 count 1 nextOffset 1 batches 1
2025-11-14T16:03:02.384772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 0:0 appended
2025-11-14T16:03:02.384776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/32] d0000000000_00000000000000000001_00000_0000000001_00000?
2025-11-14T16:03:02.384778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.384780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000001_00000_0000000001_00000?, parameters.CurOffset=1
2025-11-14T16:03:02.384813Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.385071Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 1.0/1
2025-11-14T16:03:02.385082Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 1 partNo 0
2025-11-14T16:03:02.385906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 1 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 524387 count 2 nextOffset 2 batches 1
2025-11-14T16:03:02.385922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 1:0 appended
2025-11-14T16:03:02.385928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/32] d0000000000_00000000000000000002_00000_0000000001_00000?
2025-11-14T16:03:02.385930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.385934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000002_00000_0000000001_00000?, parameters.CurOffset=2
2025-11-14T16:03:02.386012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 2.0/1
2025-11-14T16:03:02.386017Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 2 partNo 0
2025-11-14T16:03:02.386030Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 2 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 786601 count 3 nextOffset 3 batches 2
2025-11-14T16:03:02.386032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 2:0 appended
2025-11-14T16:03:02.386038Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/32] d0000000000_00000000000000000003_00000_0000000001_00000?
2025-11-14T16:03:02.386039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.386041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000003_00000_0000000001_00000?, parameters.CurOffset=3
2025-11-14T16:03:02.386104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 3.0/1
2025-11-14T16:03:02.386108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 3 partNo 0
2025-11-14T16:03:02.386634Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 3 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1048774 count 4 nextOffset 4 batches 2
2025-11-14T16:03:02.386639Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 3:0 appended
2025-11-14T16:03:02.386646Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/32] d0000000000_00000000000000000004_00000_0000000001_00000?
2025-11-14T16:03:02.386648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.386651Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000004_00000_0000000001_00000?, parameters.CurOffset=4
2025-11-14T16:03:02.386713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 4.0/1
2025-11-14T16:03:02.386717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 4 partNo 0
2025-11-14T16:03:02.386730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 4 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1310988 count 5 nextOffset 5 batches 3
2025-11-14T16:03:02.386731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 4:0 appended
2025-11-14T16:03:02.386735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/32] d0000000000_00000000000000000005_00000_0000000001_00000?
2025-11-14T16:03:02.386736Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.386738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000005_00000_0000000001_00000?, parameters.CurOffset=5
2025-11-14T16:03:02.386801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 5.0/1
2025-11-14T16:03:02.386803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 5 partNo 0
2025-11-14T16:03:02.387650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 5 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1573161 count 6 nextOffset 6 batches 3
2025-11-14T16:03:02.387667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 5:0 appended
2025-11-14T16:03:02.387671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/32] d0000000000_00000000000000000006_00000_0000000001_00000?
2025-11-14T16:03:02.387673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.387676Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000006_00000_0000000001_00000?, parameters.CurOffset=6
2025-11-14T16:03:02.387747Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 6.0/1
2025-11-14T16:03:02.387874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 6 partNo 0
2025-11-14T16:03:02.387899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 6 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 1835375 count 7 nextOffset 7 batches 4
2025-11-14T16:03:02.387902Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 6:0 appended
2025-11-14T16:03:02.387908Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/32] d0000000000_00000000000000000007_00000_0000000001_00000?
2025-11-14T16:03:02.387910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.387913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000007_00000_0000000001_00000?, parameters.CurOffset=7
2025-11-14T16:03:02.387973Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 7.0/1
2025-11-14T16:03:02.387979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 7 partNo 0
2025-11-14T16:03:02.388690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 7 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2097548 count 8 nextOffset 8 batches 4
2025-11-14T16:03:02.388703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 7:0 appended
2025-11-14T16:03:02.388709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[8/32] d0000000000_00000000000000000008_00000_0000000001_00000?
2025-11-14T16:03:02.388710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.388713Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000008_00000_0000000001_00000?, parameters.CurOffset=8
2025-11-14T16:03:02.388777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 8.0/1
2025-11-14T16:03:02.388787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 8 partNo 0
2025-11-14T16:03:02.388798Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 8 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2359762 count 9 nextOffset 9 batches 5
2025-11-14T16:03:02.388801Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 8:0 appended
2025-11-14T16:03:02.388805Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[9/32] d0000000000_00000000000000000009_00000_0000000001_00000?
2025-11-14T16:03:02.388807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.388810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000009_00000_0000000001_00000?, parameters.CurOffset=9
2025-11-14T16:03:02.388868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 9.0/1
2025-11-14T16:03:02.388876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 9 partNo 0
2025-11-14T16:03:02.389389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 9 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2621935 count 10 nextOffset 10 batches 5
2025-11-14T16:03:02.389400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 9:0 appended
2025-11-14T16:03:02.389405Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[10/32] d0000000000_00000000000000000010_00000_0000000001_00000?
2025-11-14T16:03:02.389407Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.389410Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000010_00000_0000000001_00000?, parameters.CurOffset=10
2025-11-14T16:03:02.389472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 10.0/1
2025-11-14T16:03:02.389484Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 10 partNo 0
2025-11-14T16:03:02.389498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 10 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 2884149 count 11 nextOffset 11 batches 6
2025-11-14T16:03:02.389500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 10:0 appended
2025-11-14T16:03:02.389503Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[11/32] d0000000000_00000000000000000011_00000_0000000001_00000?
2025-11-14T16:03:02.389505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.389508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000011_00000_0000000001_00000?, parameters.CurOffset=11
2025-11-14T16:03:02.389564Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 11.0/1
2025-11-14T16:03:02.389568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 11 partNo 0
2025-11-14T16:03:02.390084Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 11 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3146322 count 12 nextOffset 12 batches 6
2025-11-14T16:03:02.390087Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 11:0 appended
2025-11-14T16:03:02.390090Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[12/32] d0000000000_00000000000000000012_00000_0000000001_00000?
2025-11-14T16:03:02.390093Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.390095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000012_00000_0000000001_00000?, parameters.CurOffset=12
2025-11-14T16:03:02.390170Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 12.0/1
2025-11-14T16:03:02.390175Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 12 partNo 0
2025-11-14T16:03:02.390189Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 12 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3408536 count 13 nextOffset 13 batches 7
2025-11-14T16:03:02.390191Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 12:0 appended
2025-11-14T16:03:02.390195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[13/32] d0000000000_00000000000000000013_00000_0000000001_00000?
2025-11-14T16:03:02.390197Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.390200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000013_00000_0000000001_00000?, parameters.CurOffset=13
2025-11-14T16:03:02.390252Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 13.0/1
2025-11-14T16:03:02.390257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 13 partNo 0
2025-11-14T16:03:02.390730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 13 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3670709 count 14 nextOffset 14 batches 7
2025-11-14T16:03:02.390734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 13:0 appended
2025-11-14T16:03:02.390738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[14/32] d0000000000_00000000000000000014_00000_0000000001_00000?
2025-11-14T16:03:02.390740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.390743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000014_00000_0000000001_00000?, parameters.CurOffset=14
2025-11-14T16:03:02.390819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 14.0/1
2025-11-14T16:03:02.390823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 14 partNo 0
2025-11-14T16:03:02.390835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 14 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 3932923 count 15 nextOffset 15 batches 8
2025-11-14T16:03:02.390837Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 14:0 appended
2025-11-14T16:03:02.390840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[15/32] d0000000000_00000000000000000015_00000_0000000001_00000?
2025-11-14T16:03:02.390842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.390845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000015_00000_0000000001_00000?, parameters.CurOffset=15
2025-11-14T16:03:02.390899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 15.0/1
2025-11-14T16:03:02.390903Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 15 partNo 0
2025-11-14T16:03:02.391291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 15 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4195096 count 16 nextOffset 16 batches 8
2025-11-14T16:03:02.391297Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 15:0 appended
2025-11-14T16:03:02.391302Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[16/32] d0000000000_00000000000000000016_00000_0000000001_00000?
2025-11-14T16:03:02.391304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.391308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000016_00000_0000000001_00000?, parameters.CurOffset=16
2025-11-14T16:03:02.391383Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 16.0/1
2025-11-14T16:03:02.391388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 16 partNo 0
2025-11-14T16:03:02.391421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 16 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4457310 count 17 nextOffset 17 batches 9
2025-11-14T16:03:02.391425Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 16:0 appended
2025-11-14T16:03:02.391428Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[17/32] d0000000000_00000000000000000017_00000_0000000001_00000?
2025-11-14T16:03:02.391430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.391433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000017_00000_0000000001_00000?, parameters.CurOffset=17
2025-11-14T16:03:02.391495Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 17.0/1
2025-11-14T16:03:02.391500Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 17 partNo 0
2025-11-14T16:03:02.391818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 17 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4719483 count 18 nextOffset 18 batches 9
2025-11-14T16:03:02.391822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 17:0 appended
2025-11-14T16:03:02.391827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[18/32] d0000000000_00000000000000000018_00000_0000000001_00000?
2025-11-14T16:03:02.391829Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.391832Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000018_00000_0000000001_00000?, parameters.CurOffset=18
2025-11-14T16:03:02.391895Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 18.0/1
2025-11-14T16:03:02.391899Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 18 partNo 0
2025-11-14T16:03:02.391909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 18 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 4981697 count 19 nextOffset 19 batches 10
2025-11-14T16:03:02.391911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 18:0 appended
2025-11-14T16:03:02.391915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[19/32] d0000000000_00000000000000000019_00000_0000000001_00000?
2025-11-14T16:03:02.391917Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.391920Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000019_00000_0000000001_00000?, parameters.CurOffset=19
2025-11-14T16:03:02.391976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 19.0/1
2025-11-14T16:03:02.391979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 19 partNo 0
2025-11-14T16:03:02.392530Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 19 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 5243870 count 20 nextOffset 20 batches 10
2025-11-14T16:03:02.392536Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 19:0 appended
2025-11-14T16:03:02.392543Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[20/32] d0000000000_00000000000000000020_00000_0000000001_00000?
2025-11-14T16:03:02.392545Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.392549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000020_00000_0000000001_00000?, parameters.CurOffset=20
2025-11-14T16:03:02.392615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 20.0/1
2025-11-14T16:03:02.392618Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 20 partNo 0
2025-11-14T16:03:02.392626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 20 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 5506084 count 21 nextOffset 21 batches 11
2025-11-14T16:03:02.392628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 20:0 appended
2025-11-14T16:03:02.392630Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[21/32] d0000000000_00000000000000000021_00000_0000000001_00000?
2025-11-14T16:03:02.392631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.392635Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000021_00000_0000000001_00000?, parameters.CurOffset=21
2025-11-14T16:03:02.392687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 21.0/1
2025-11-14T16:03:02.392690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 21 partNo 0
2025-11-14T16:03:02.393686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 21 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 5768257 count 22 nextOffset 22 batches 11
2025-11-14T16:03:02.393697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 21:0 appended
2025-11-14T16:03:02.393703Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[22/32] d0000000000_00000000000000000022_00000_0000000001_00000?
2025-11-14T16:03:02.393705Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.393709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000022_00000_0000000001_00000?, parameters.CurOffset=22
2025-11-14T16:03:02.393780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 22.0/1
2025-11-14T16:03:02.393790Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 22 partNo 0
2025-11-14T16:03:02.393803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 22 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 6030471 count 23 nextOffset 23 batches 12
2025-11-14T16:03:02.393810Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 22:0 appended
2025-11-14T16:03:02.393814Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[23/32] d0000000000_00000000000000000023_00000_0000000001_00000?
2025-11-14T16:03:02.393817Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.393820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000023_00000_0000000001_00000?, parameters.CurOffset=23
2025-11-14T16:03:02.393880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 23.0/1
2025-11-14T16:03:02.393888Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 23 partNo 0
2025-11-14T16:03:02.394377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 23 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 6292644 count 24 nextOffset 24 batches 12
2025-11-14T16:03:02.394386Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 23:0 appended
2025-11-14T16:03:02.394390Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[24/32] d0000000000_00000000000000000024_00000_0000000001_00000?
2025-11-14T16:03:02.394392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.394395Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000024_00000_0000000001_00000?, parameters.CurOffset=24
2025-11-14T16:03:02.394463Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 24.0/1
2025-11-14T16:03:02.394474Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 24 partNo 0
2025-11-14T16:03:02.394485Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 24 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 6554858 count 25 nextOffset 25 batches 13
2025-11-14T16:03:02.394487Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 24:0 appended
2025-11-14T16:03:02.394490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[25/32] d0000000000_00000000000000000025_00000_0000000001_00000?
2025-11-14T16:03:02.394492Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.394494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000025_00000_0000000001_00000?, parameters.CurOffset=25
2025-11-14T16:03:02.394558Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 25.0/1
2025-11-14T16:03:02.394567Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 25 partNo 0
2025-11-14T16:03:02.395013Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 25 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 6817031 count 26 nextOffset 26 batches 13
2025-11-14T16:03:02.395027Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 25:0 appended
2025-11-14T16:03:02.395032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[26/32] d0000000000_00000000000000000026_00000_0000000001_00000?
2025-11-14T16:03:02.395033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.395035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000026_00000_0000000001_00000?, parameters.CurOffset=26
2025-11-14T16:03:02.395101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 26.0/1
2025-11-14T16:03:02.395104Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 26 partNo 0
2025-11-14T16:03:02.395113Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 26 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 7079245 count 27 nextOffset 27 batches 14
2025-11-14T16:03:02.395115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 26:0 appended
2025-11-14T16:03:02.395118Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[27/32] d0000000000_00000000000000000027_00000_0000000001_00000?
2025-11-14T16:03:02.395120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.395122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000027_00000_0000000001_00000?, parameters.CurOffset=27
2025-11-14T16:03:02.395182Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 27.0/1
2025-11-14T16:03:02.395185Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 27 partNo 0
2025-11-14T16:03:02.395675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 27 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 7341418 count 28 nextOffset 28 batches 14
2025-11-14T16:03:02.395680Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 27:0 appended
2025-11-14T16:03:02.395686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[28/32] d0000000000_00000000000000000028_00000_0000000001_00000?
2025-11-14T16:03:02.395687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.395690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000028_00000_0000000001_00000?, parameters.CurOffset=28
2025-11-14T16:03:02.395758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 28.0/1
2025-11-14T16:03:02.395763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 28 partNo 0
2025-11-14T16:03:02.395774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 28 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 7603632 count 29 nextOffset 29 batches 15
2025-11-14T16:03:02.395776Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 28:0 appended
2025-11-14T16:03:02.395780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[29/32] d0000000000_00000000000000000029_00000_0000000001_00000?
2025-11-14T16:03:02.395781Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.395784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000029_00000_0000000001_00000?, parameters.CurOffset=29
2025-11-14T16:03:02.395843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 29.0/1
2025-11-14T16:03:02.395849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 29 partNo 0
2025-11-14T16:03:02.396349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 29 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 7865805 count 30 nextOffset 30 batches 15
2025-11-14T16:03:02.396352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 29:0 appended
2025-11-14T16:03:02.396357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[30/32] d0000000000_00000000000000000030_00000_0000000001_00000?
2025-11-14T16:03:02.396359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.396361Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000030_00000_0000000001_00000?, parameters.CurOffset=30
2025-11-14T16:03:02.396429Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 30.0/1
2025-11-14T16:03:02.396433Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 30 partNo 0
2025-11-14T16:03:02.396441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 30 partNo 0 FormedBlobsCount 0 NewHead: Offset 0 PartNo 0 PackedSize 8128019 count 31 nextOffset 31 batches 16
2025-11-14T16:03:02.396443Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 30:0 appended
2025-11-14T16:03:02.396447Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[31/32] d0000000000_00000000000000000031_00000_0000000001_00000?
2025-11-14T16:03:02.396448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.396451Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000031_00000_0000000001_00000?, parameters.CurOffset=31
2025-11-14T16:03:02.396509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 31.0/1
2025-11-14T16:03:02.396512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 31 partNo 0
2025-11-14T16:03:02.404541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 31 partNo 0 result is x0000000000_00000000000000000000_00000_0000000031_00000 size 8128009
2025-11-14T16:03:02.404566Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1104: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000000_00000_0000000031_00000 new key d0000000000_00000000000000000000_00000_0000000031_00000 size 8128009 WTime 1763136182404
2025-11-14T16:03:02.404602Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 31 partNo 0 FormedBlobsCount 1 NewHead: Offset 31 PartNo 0 PackedSize 262214 count 1 nextOffset 32 batches 1
2025-11-14T16:03:02.404608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 31:0 appended
2025-11-14T16:03:02.404738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 31,1 HeadOffset 0 endOffset 0 curOffset 32 d0000000000_00000000000000000031_00000_0000000001_00000| size 262204 WTime 1763136182404
2025-11-14T16:03:02.404830Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951336:3461], now have 1 active actors on pipe
2025-11-14T16:03:02.404856Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.404862Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.404875Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.404890Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 0 partNo 0 count 31 size 8128009
2025-11-14T16:03:02.404898Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 31 partNo 0 count 1 size 262204
2025-11-14T16:03:02.404951Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|7305e290-256a31d7-3427ef4e-e92bce3a_32 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.404959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.404963Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.404968Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.404971Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.404975Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.404985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.404988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.404991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.405006Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.405045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.405336Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.405704Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951336:3461] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|7305e290-256a31d7-3427ef4e-e92bce3a_32"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|7305e290-256a31d7-3427ef4e-e92bce3a_32"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 32
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.408877Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.409448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951341:3463], now have 1 active actors on pipe
2025-11-14T16:03:02.409855Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.409870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.409891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 32 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.409903Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.409917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.409921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.409933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.409935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.409940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.409969Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 32 partNo 0
2025-11-14T16:03:02.410001Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 32 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 262214 count 1 nextOffset 33 batches 1
2025-11-14T16:03:02.410007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.410010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.410013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.410141Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 32,1 HeadOffset 32 endOffset 32 curOffset 33 d0000000000_00000000000000000032_00000_0000000001_00000? size 262204 WTime 1763136182409
2025-11-14T16:03:02.410178Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.410188Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 32 partNo 0 count 1 size 262204
2025-11-14T16:03:02.417629Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 0 count 31 size 8128009 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.417648Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 31 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.417670Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 32 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.417680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.417693Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 31 parts 0 suffix '0' size 8128009
2025-11-14T16:03:02.417701Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 0 suffix '124' size 262204
2025-11-14T16:03:02.417708Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 32 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.417791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2289: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-14T16:03:02.417800Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.417805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.417828Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.417843Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.417858Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 32, partNo: 0, Offset: 32 is stored on disk
2025-11-14T16:03:02.417887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.417894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.417896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.417899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.417902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.417950Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.417962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2289: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-14T16:03:02.417976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.417999Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.418010Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000000_00000_0000000001_00000?(+) to d0000000000_00000000000000000000_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418016Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000001_00000_0000000001_00000?(+) to d0000000000_00000000000000000001_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418019Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000002_00000_0000000001_00000?(+) to d0000000000_00000000000000000002_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418022Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000003_00000_0000000001_00000?(+) to d0000000000_00000000000000000003_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418024Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000004_00000_0000000001_00000?(+) to d0000000000_00000000000000000004_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418029Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000005_00000_0000000001_00000?(+) to d0000000000_00000000000000000005_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418031Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000006_00000_0000000001_00000?(+) to d0000000000_00000000000000000006_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418033Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000007_00000_0000000001_00000?(+) to d0000000000_00000000000000000007_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418036Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000008_00000_0000000001_00000?(+) to d0000000000_00000000000000000008_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418039Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000009_00000_0000000001_00000?(+) to d0000000000_00000000000000000009_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418041Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000010_00000_0000000001_00000?(+) to d0000000000_00000000000000000010_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418043Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000011_00000_0000000001_00000?(+) to d0000000000_00000000000000000011_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418046Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000012_00000_0000000001_00000?(+) to d0000000000_00000000000000000012_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418049Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000013_00000_0000000001_00000?(+) to d0000000000_00000000000000000013_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418051Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000014_00000_0000000001_00000?(+) to d0000000000_00000000000000000014_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418053Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000015_00000_0000000001_00000?(+) to d0000000000_00000000000000000015_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418056Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000016_00000_0000000001_00000?(+) to d0000000000_00000000000000000016_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418059Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000017_00000_0000000001_00000?(+) to d0000000000_00000000000000000017_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418062Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000018_00000_0000000001_00000?(+) to d0000000000_00000000000000000018_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418065Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000019_00000_0000000001_00000?(+) to d0000000000_00000000000000000019_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418067Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000020_00000_0000000001_00000?(+) to d0000000000_00000000000000000020_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418070Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000021_00000_0000000001_00000?(+) to d0000000000_00000000000000000021_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418072Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000022_00000_0000000001_00000?(+) to d0000000000_00000000000000000022_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418076Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000023_00000_0000000001_00000?(+) to d0000000000_00000000000000000023_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418078Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000024_00000_0000000001_00000?(+) to d0000000000_00000000000000000024_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418081Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000025_00000_0000000001_00000?(+) to d0000000000_00000000000000000025_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418083Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000026_00000_0000000001_00000?(+) to d0000000000_00000000000000000026_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418086Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000027_00000_0000000001_00000?(+) to d0000000000_00000000000000000027_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418088Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000028_00000_0000000001_00000?(+) to d0000000000_00000000000000000028_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418091Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000029_00000_0000000001_00000?(+) to d0000000000_00000000000000000029_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418093Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000030_00000_0000000001_00000?(+) to d0000000000_00000000000000000030_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418096Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000031_00000_0000000001_00000?(+) to d0000000000_00000000000000000031_00000_0000000001_00000?(+)
2025-11-14T16:03:02.418100Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 32
    Offset: 32
    WriteTimestampMS: 1763136182409
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 6
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-14T16:03:02.418490Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951341:3463] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.419190Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.419244Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 0 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419268Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 1 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 2 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419282Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 3 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 4 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419297Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 5 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419305Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 6 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419310Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 7 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419317Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 8 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419326Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 9 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419341Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 10 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419348Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 11 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419352Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 12 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419357Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 13 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 14 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419370Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 15 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419377Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 16 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419385Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 17 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419390Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 18 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419393Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 19 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419396Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 20 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 21 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419412Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 22 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419416Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 23 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419419Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 24 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419422Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 25 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419427Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 26 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419431Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 27 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419434Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 28 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419439Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 29 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419444Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 30 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419448Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 31 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.419470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.419482Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.419490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.419492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.419494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.419493Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.419498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.419501Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 1 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419505Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 2 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419506Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.419508Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 3 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419512Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 4 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419517Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 5 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419521Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 6 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419524Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 7 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419528Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 8 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419531Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 9 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419535Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 10 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419538Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 11 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419542Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 12 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419547Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 13 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419554Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 14 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419558Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 15 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419562Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 16 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419565Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 17 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419569Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 18 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419572Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 19 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419576Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 20 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419580Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 21 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419584Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 22 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419587Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 23 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419590Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 24 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419594Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 25 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419598Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 26 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419602Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 27 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419606Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 28 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419610Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 29 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419613Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 30 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419617Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.419883Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951347:3467], now have 1 active actors on pipe
2025-11-14T16:03:02.420066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.420086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.420122Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|d081d51-ad3bb74e-f5d44899-ee7f5890_33 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.420134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.420143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.420148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.420151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.420156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.420166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.420172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.420175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.420187Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.420334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.420394Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.420690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951347:3467] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|d081d51-ad3bb74e-f5d44899-ee7f5890_33"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|d081d51-ad3bb74e-f5d44899-ee7f5890_33"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 33
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.423281Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.423801Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951351:3469], now have 1 active actors on pipe
2025-11-14T16:03:02.424165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.424181Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.424213Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 33 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.424231Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.424247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.424255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.424263Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.424264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.424267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.424293Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 33 partNo 0
2025-11-14T16:03:02.424326Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 33 partNo 0 FormedBlobsCount 0 NewHead: Offset 33 PartNo 0 PackedSize 262214 count 1 nextOffset 34 batches 1
2025-11-14T16:03:02.424334Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.424336Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.424338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.424435Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 33,1 HeadOffset 33 endOffset 33 curOffset 34 d0000000000_00000000000000000033_00000_0000000001_00000? size 262204 WTime 1763136182424
2025-11-14T16:03:02.424473Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.424485Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 33 partNo 0 count 1 size 262204
2025-11-14T16:03:02.438556Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 33 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.438594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.438617Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.438628Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 33 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.438632Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.438642Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 33, partNo: 0, Offset: 33 is stored on disk
2025-11-14T16:03:02.438684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.438695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.438700Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.438707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.438709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.438715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.438728Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.438852Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.439248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951351:3469] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 33
    Offset: 33
    WriteTimestampMS: 1763136182424
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 14
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.439990Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.440564Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951355:3471], now have 1 active actors on pipe
2025-11-14T16:03:02.440699Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.440713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.440877Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.440748Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|566b4c1e-1a608ede-8ba5adcb-d083e584_34 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.440763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.440766Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.440772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.440774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.440778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.440787Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.440790Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.440792Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.440805Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.440818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
2025-11-14T16:03:02.441194Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951355:3471] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|566b4c1e-1a608ede-8ba5adcb-d083e584_34"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|566b4c1e-1a608ede-8ba5adcb-d083e584_34"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 34
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.443103Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.443546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951358:3472], now have 1 active actors on pipe
2025-11-14T16:03:02.443938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.443962Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.443980Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 34 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.443991Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.444007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.444010Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.444018Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.444020Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.444024Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.444047Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 34 partNo 0
2025-11-14T16:03:02.444086Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 34 partNo 0 FormedBlobsCount 0 NewHead: Offset 34 PartNo 0 PackedSize 262214 count 1 nextOffset 35 batches 1
2025-11-14T16:03:02.444097Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.444112Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.444117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.444224Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 34,1 HeadOffset 34 endOffset 34 curOffset 35 d0000000000_00000000000000000034_00000_0000000001_00000? size 262204 WTime 1763136182444
2025-11-14T16:03:02.444261Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.444276Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 34 partNo 0 count 1 size 262204
2025-11-14T16:03:02.447938Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 34 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.447966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.447987Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.447994Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 34 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.447997Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.448007Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 34, partNo: 0, Offset: 34 is stored on disk
2025-11-14T16:03:02.448040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.448057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.448065Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.448067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.448071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.448077Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.448087Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.448171Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.448470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951358:3472] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 34
    Offset: 34
    WriteTimestampMS: 1763136182443
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.449156Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.449604Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951362:3474], now have 1 active actors on pipe
2025-11-14T16:03:02.449691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.449702Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.449734Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|cf4d9a51-b500250c-e2ac37de-fa615377_35 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.449746Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.449750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.449754Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.449756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.449760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.449768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.449772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.449777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.449787Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.449799Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.449850Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.450067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951362:3474] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|cf4d9a51-b500250c-e2ac37de-fa615377_35"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|cf4d9a51-b500250c-e2ac37de-fa615377_35"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 35
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.452168Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.452640Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951365:3475], now have 1 active actors on pipe
2025-11-14T16:03:02.453081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.453096Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.453116Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 35 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.453131Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.453148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.453151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.453160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.453162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.453180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.453205Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 35 partNo 0
2025-11-14T16:03:02.453258Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 35 partNo 0 FormedBlobsCount 0 NewHead: Offset 35 PartNo 0 PackedSize 262214 count 1 nextOffset 36 batches 1
2025-11-14T16:03:02.453268Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.453271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.453275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.453381Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 35,1 HeadOffset 35 endOffset 35 curOffset 36 d0000000000_00000000000000000035_00000_0000000001_00000? size 262204 WTime 1763136182452
2025-11-14T16:03:02.453421Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.453436Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 35 partNo 0 count 1 size 262204
2025-11-14T16:03:02.455522Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 35 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.455556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.455576Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 35 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.455578Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.455588Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.455597Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 35, partNo: 0, Offset: 35 is stored on disk
2025-11-14T16:03:02.455635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.455644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.455647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.455653Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.455655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.455662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.455689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.455770Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 35
    Offset: 35
    WriteTimestampMS: 1763136182452
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.457418Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.458308Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|5459b3e1-94f8c6bc-5b05750d-ac306b9d_36"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|5459b3e1-94f8c6bc-5b05750d-ac306b9d_36"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 36
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.457048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951365:3475] destroyed
2025-11-14T16:03:02.457982Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951369:3477], now have 1 active actors on pipe
2025-11-14T16:03:02.458120Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.458127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.458166Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|5459b3e1-94f8c6bc-5b05750d-ac306b9d_36 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.458176Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.458179Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.458185Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.458187Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.458191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.458202Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.458205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.458211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.458227Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.458242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.458625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951369:3477] destroyed
2025-11-14T16:03:02.461099Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.461517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951372:3478], now have 1 active actors on pipe
2025-11-14T16:03:02.462002Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.462030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.462058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 36 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.462081Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.462105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.462119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.462132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.462141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.462146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.462180Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 36 partNo 0
2025-11-14T16:03:02.462228Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 36 partNo 0 FormedBlobsCount 0 NewHead: Offset 36 PartNo 0 PackedSize 262214 count 1 nextOffset 37 batches 1
2025-11-14T16:03:02.462241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.462250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.462255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.462402Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 36,1 HeadOffset 36 endOffset 36 curOffset 37 d0000000000_00000000000000000036_00000_0000000001_00000? size 262204 WTime 1763136182462
2025-11-14T16:03:02.462462Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.462480Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 36 partNo 0 count 1 size 262204
2025-11-14T16:03:02.469383Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 36 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.469429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.469443Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 36 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.469457Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.469473Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.469487Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 36, partNo: 0, Offset: 36 is stored on disk
2025-11-14T16:03:02.469543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.469554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.469557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.469561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.469564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.469578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.469597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.469756Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.470165Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951372:3478] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 36
    Offset: 36
    WriteTimestampMS: 1763136182462
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 7
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.473937Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.476871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951377:3481], now have 1 active actors on pipe
2025-11-14T16:03:02.476965Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.476979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.477035Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|e2ae2191-5f29adf7-9cedf076-9aada641_37 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.477053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.477061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.477070Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.477073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.477078Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.477092Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.477100Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.477103Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.477122Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.477145Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.477604Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.477873Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951377:3481] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|e2ae2191-5f29adf7-9cedf076-9aada641_37"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|e2ae2191-5f29adf7-9cedf076-9aada641_37"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 37
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.484488Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.485089Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951380:3482], now have 1 active actors on pipe
2025-11-14T16:03:02.485844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.485876Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.485905Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 37 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.485920Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.485939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.485942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.485953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.485955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.485962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.485994Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 37 partNo 0
2025-11-14T16:03:02.486038Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 37 partNo 0 FormedBlobsCount 0 NewHead: Offset 37 PartNo 0 PackedSize 262214 count 1 nextOffset 38 batches 1
2025-11-14T16:03:02.486046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.486049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.486053Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.486180Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 37,1 HeadOffset 37 endOffset 37 curOffset 38 d0000000000_00000000000000000037_00000_0000000001_00000? size 262204 WTime 1763136182485
2025-11-14T16:03:02.486218Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.486230Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 37 partNo 0 count 1 size 262204
2025-11-14T16:03:02.492154Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.491853Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 37 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.491891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.491916Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.491927Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.491940Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 37, partNo: 0, Offset: 37 is stored on disk
2025-11-14T16:03:02.491983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.491987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.491990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.491993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.491995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.492004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.492016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.492081Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 37 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.492597Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951380:3482] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 37
    Offset: 37
    WriteTimestampMS: 1763136182485
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 6
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.496368Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.499005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951384:3484], now have 1 active actors on pipe
2025-11-14T16:03:02.499058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.499069Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.499141Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|6c404c50-8129f2b9-8d865dfe-fdf91185_38 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.499169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.499174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.499181Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.499184Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.499189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.499217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.499223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.499226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.499244Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.499263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.499339Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|6c404c50-8129f2b9-8d865dfe-fdf91185_38"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
2025-11-14T16:03:02.499836Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951384:3484] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|6c404c50-8129f2b9-8d865dfe-fdf91185_38"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 38
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.502441Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.503232Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951387:3485], now have 1 active actors on pipe
2025-11-14T16:03:02.503545Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.503560Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.503588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 38 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.503605Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.503629Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.503633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.503647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.503650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.503659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.503700Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 38 partNo 0
2025-11-14T16:03:02.503748Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 38 partNo 0 FormedBlobsCount 0 NewHead: Offset 38 PartNo 0 PackedSize 262214 count 1 nextOffset 39 batches 1
2025-11-14T16:03:02.503759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.503769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.503773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.503908Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 38,1 HeadOffset 38 endOffset 38 curOffset 39 d0000000000_00000000000000000038_00000_0000000001_00000? size 262204 WTime 1763136182503
2025-11-14T16:03:02.503947Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.503960Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 38 partNo 0 count 1 size 262204
2025-11-14T16:03:02.506712Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 38 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.506739Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.506761Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.506772Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.506783Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 38, partNo: 0, Offset: 38 is stored on disk
2025-11-14T16:03:02.506816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.506819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.506821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.506825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.506827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.506836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.506847Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.506909Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 38 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.506989Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.507533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951387:3485] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 38
    Offset: 38
    WriteTimestampMS: 1763136182503
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.509293Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.509750Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951391:3487], now have 1 active actors on pipe
2025-11-14T16:03:02.509858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.509870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.509904Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|d80cf049-8a17aec9-4888a3b6-5ec71ca8_39 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.509918Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.509921Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.509926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.509927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.509931Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.509942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.509945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.509947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.509961Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.509978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.510160Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|d80cf049-8a17aec9-4888a3b6-5ec71ca8_39"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|d80cf049-8a17aec9-4888a3b6-5ec71ca8_39"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 39
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.511569Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951391:3487] destroyed
2025-11-14T16:03:02.514115Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.514650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951394:3488], now have 1 active actors on pipe
2025-11-14T16:03:02.514993Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.515003Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.515027Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 39 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.515040Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.515071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.515075Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.515086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.515089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.515094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.515122Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 39 partNo 0
2025-11-14T16:03:02.515154Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 39 partNo 0 FormedBlobsCount 0 NewHead: Offset 39 PartNo 0 PackedSize 262214 count 1 nextOffset 40 batches 1
2025-11-14T16:03:02.515160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.515162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.515164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.515252Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 39,1 HeadOffset 39 endOffset 39 curOffset 40 d0000000000_00000000000000000039_00000_0000000001_00000? size 262204 WTime 1763136182514
2025-11-14T16:03:02.515279Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.515288Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 39 partNo 0 count 1 size 262204
2025-11-14T16:03:02.517904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2289: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-14T16:03:02.521418Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 39 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.521452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.521469Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 39 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.521486Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.521498Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.521509Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 39, partNo: 0, Offset: 39 is stored on disk
2025-11-14T16:03:02.521546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.521549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.521551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.521554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.521557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.521565Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.521577Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.521667Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.522028Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951394:3488] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 39
    Offset: 39
    WriteTimestampMS: 1763136182514
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 6
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.522765Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.523276Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951399:3491], now have 1 active actors on pipe
2025-11-14T16:03:02.523382Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.523394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.523435Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|6c4e8fc6-706a1f6e-e8925325-5945b6a0_40 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.523448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.523452Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.523458Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.523462Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.523466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.523476Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.523479Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.523482Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.523496Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.523508Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.523984Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|6c4e8fc6-706a1f6e-e8925325-5945b6a0_40"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
2025-11-14T16:03:02.524371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951399:3491] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|6c4e8fc6-706a1f6e-e8925325-5945b6a0_40"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 40
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.526767Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.527417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951402:3492], now have 1 active actors on pipe
2025-11-14T16:03:02.528127Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.528146Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.528171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 40 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.528189Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.528210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.528218Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.528228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.528235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.528241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.528269Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 40 partNo 0
2025-11-14T16:03:02.528306Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 40 partNo 0 FormedBlobsCount 0 NewHead: Offset 40 PartNo 0 PackedSize 262214 count 1 nextOffset 41 batches 1
2025-11-14T16:03:02.528316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.528335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.528342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.528453Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 40,1 HeadOffset 40 endOffset 40 curOffset 41 d0000000000_00000000000000000040_00000_0000000001_00000? size 262204 WTime 1763136182528
2025-11-14T16:03:02.528493Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.528509Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 40 partNo 0 count 1 size 262204
2025-11-14T16:03:02.537688Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 40 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.537725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.537750Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.537763Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.537775Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 40, partNo: 0, Offset: 40 is stored on disk
2025-11-14T16:03:02.537827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.537835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.537838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.537841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.537843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.537853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.537870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.537955Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 40 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.538047Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 40
    Offset: 40
    WriteTimestampMS: 1763136182528
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 9
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.538453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951402:3492] destroyed
2025-11-14T16:03:02.539060Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.539629Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951406:3494], now have 1 active actors on pipe
2025-11-14T16:03:02.539729Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.539742Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.539781Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|75b639a6-ce110f26-f2d970d2-84a42138_41 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.539793Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.539799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.539804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.539806Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.539810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.539819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.539822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.539829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.539841Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.539857Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.539916Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.540159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951406:3494] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|75b639a6-ce110f26-f2d970d2-84a42138_41"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|75b639a6-ce110f26-f2d970d2-84a42138_41"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 41
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.542407Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.542804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951409:3495], now have 1 active actors on pipe
2025-11-14T16:03:02.543236Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.543252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.543273Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 41 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.543289Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.543307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.543318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.543327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.543329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.543335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.543364Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 41 partNo 0
2025-11-14T16:03:02.543393Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 41 partNo 0 FormedBlobsCount 0 NewHead: Offset 41 PartNo 0 PackedSize 262214 count 1 nextOffset 42 batches 1
2025-11-14T16:03:02.543405Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.543408Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.543411Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.543531Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 41,1 HeadOffset 41 endOffset 41 curOffset 42 d0000000000_00000000000000000041_00000_0000000001_00000? size 262204 WTime 1763136182543
2025-11-14T16:03:02.543565Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.543577Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 41 partNo 0 count 1 size 262204
2025-11-14T16:03:02.548511Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 41 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.548546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.548570Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.548582Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.548600Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 41, partNo: 0, Offset: 41 is stored on disk
2025-11-14T16:03:02.548603Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 41 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.548637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.548641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.548643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.548646Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.548648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.548656Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.548669Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.548774Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 41
    Offset: 41
    WriteTimestampMS: 1763136182543
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
2025-11-14T16:03:02.549235Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951409:3495] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.549875Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.550443Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951413:3497], now have 1 active actors on pipe
2025-11-14T16:03:02.550562Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.550570Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.550597Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|471bee21-5a28663d-7a2df90b-c3a42b06_42 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.550606Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.550608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.550614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.550617Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.550620Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.550628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.550630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.550633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.550644Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.550657Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.550803Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.551121Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951413:3497] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|471bee21-5a28663d-7a2df90b-c3a42b06_42"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|471bee21-5a28663d-7a2df90b-c3a42b06_42"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 42
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.553750Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.554285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951416:3498], now have 1 active actors on pipe
2025-11-14T16:03:02.554775Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.554792Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.554812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 42 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.554824Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.554838Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.554842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.554851Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.554855Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.554861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.554888Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 42 partNo 0
2025-11-14T16:03:02.554918Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 42 partNo 0 FormedBlobsCount 0 NewHead: Offset 42 PartNo 0 PackedSize 262214 count 1 nextOffset 43 batches 1
2025-11-14T16:03:02.554925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.554927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.554930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.555041Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 42,1 HeadOffset 42 endOffset 42 curOffset 43 d0000000000_00000000000000000042_00000_0000000001_00000? size 262204 WTime 1763136182554
2025-11-14T16:03:02.555075Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.555087Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 42 partNo 0 count 1 size 262204
2025-11-14T16:03:02.558513Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 42 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.558546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.558567Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.558572Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 42 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.558582Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.558593Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 42, partNo: 0, Offset: 42 is stored on disk
2025-11-14T16:03:02.558625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.558634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.558637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.558639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.558641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.558801Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.558650Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.558682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.559111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951416:3498] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 42
    Offset: 42
    WriteTimestampMS: 1763136182554
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.561825Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.562303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951420:3500], now have 1 active actors on pipe
2025-11-14T16:03:02.562394Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.562413Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.562464Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f582530c-6982f27d-b6f68726-c69867ee_43 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.562480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.562484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.562490Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.562493Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.562497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.562508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.562511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.562515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.562530Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.562546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.562645Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f582530c-6982f27d-b6f68726-c69867ee_43"
    Status: Active
    SeqNo: 0
  }
}
2025-11-14T16:03:02.562938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951420:3500] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f582530c-6982f27d-b6f68726-c69867ee_43"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 43
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.570512Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.571104Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951423:3501], now have 1 active actors on pipe
2025-11-14T16:03:02.571595Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.571615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.571642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 43 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.571657Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.571678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.571682Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.571693Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.571705Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.571711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.571740Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 43 partNo 0
2025-11-14T16:03:02.571784Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 43 partNo 0 FormedBlobsCount 0 NewHead: Offset 43 PartNo 0 PackedSize 262214 count 1 nextOffset 44 batches 1
2025-11-14T16:03:02.571796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.571799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.571802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.571926Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 43,1 HeadOffset 43 endOffset 43 curOffset 44 d0000000000_00000000000000000043_00000_0000000001_00000? size 262204 WTime 1763136182571
2025-11-14T16:03:02.571970Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.571989Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 43 partNo 0 count 1 size 262204
2025-11-14T16:03:02.574759Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 43 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.574783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.574804Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.574815Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.574826Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 43, partNo: 0, Offset: 43 is stored on disk
2025-11-14T16:03:02.574860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.574863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.574865Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.574868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.574870Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.574879Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.574891Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.574923Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 43 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.574999Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.575478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951423:3501] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 43
    Offset: 43
    WriteTimestampMS: 1763136182571
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.576695Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.577583Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951427:3503], now have 1 active actors on pipe
2025-11-14T16:03:02.577607Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.577614Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.577646Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|1a3673fe-2e6d285b-dcab9a9d-55361c0b_44 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.577654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.577658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.577665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.577667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.577671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.577680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.577683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.577685Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.577698Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.577712Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.577779Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|1a3673fe-2e6d285b-dcab9a9d-55361c0b_44"
    Status: Active
    SeqNo: 0
  }
}
2025-11-14T16:03:02.578129Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951427:3503] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|1a3673fe-2e6d285b-dcab9a9d-55361c0b_44"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 44
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.580441Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.580831Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951430:3504], now have 1 active actors on pipe
2025-11-14T16:03:02.581313Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.581327Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.581348Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 44 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.581363Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.581376Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.581379Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.581389Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.581390Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.581395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.581419Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 44 partNo 0
2025-11-14T16:03:02.581448Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 44 partNo 0 FormedBlobsCount 0 NewHead: Offset 44 PartNo 0 PackedSize 262214 count 1 nextOffset 45 batches 1
2025-11-14T16:03:02.581455Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.581457Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.581460Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.581570Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 44,1 HeadOffset 44 endOffset 44 curOffset 45 d0000000000_00000000000000000044_00000_0000000001_00000? size 262204 WTime 1763136182581
2025-11-14T16:03:02.581605Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.581616Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 44 partNo 0 count 1 size 262204
2025-11-14T16:03:02.584012Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 44 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.584062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.584078Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 44 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.584088Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.584103Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.584127Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 44, partNo: 0, Offset: 44 is stored on disk
2025-11-14T16:03:02.584165Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.584168Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.584169Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.584172Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.584175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.584183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.584199Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.584327Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 44
    Offset: 44
    WriteTimestampMS: 1763136182580
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-14T16:03:02.584678Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951430:3504] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.585328Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.585639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951434:3506], now have 1 active actors on pipe
2025-11-14T16:03:02.585838Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.585844Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.585870Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|410dd8f4-5b804441-dff89ad4-9aceebfb_45 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.585875Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.585877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.585880Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.585881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.585883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.585891Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.585892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.585894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.585901Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.585909Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.585972Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|410dd8f4-5b804441-dff89ad4-9aceebfb_45"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|410dd8f4-5b804441-dff89ad4-9aceebfb_45"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 45
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.586261Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951434:3506] destroyed
2025-11-14T16:03:02.588283Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.588673Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951437:3507], now have 1 active actors on pipe
2025-11-14T16:03:02.589141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.589153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.589182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 45 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.589191Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.589201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.589204Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.589212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.589215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.589219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.589243Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 45 partNo 0
2025-11-14T16:03:02.589274Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 45 partNo 0 FormedBlobsCount 0 NewHead: Offset 45 PartNo 0 PackedSize 262214 count 1 nextOffset 46 batches 1
2025-11-14T16:03:02.589280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.589283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.589286Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.589394Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 45,1 HeadOffset 45 endOffset 45 curOffset 46 d0000000000_00000000000000000045_00000_0000000001_00000? size 262204 WTime 1763136182589
2025-11-14T16:03:02.589427Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.589439Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 45 partNo 0 count 1 size 262204
2025-11-14T16:03:02.591742Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 45 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.591767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.591786Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.591789Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 45 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.591795Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.591804Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 45, partNo: 0, Offset: 45 is stored on disk
2025-11-14T16:03:02.591831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.591841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.591843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.591845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.591847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.591854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.591863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.591969Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 45
    Offset: 45
    WriteTimestampMS: 1763136182589
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 0
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.592365Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951437:3507] destroyed
2025-11-14T16:03:02.592935Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.593333Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951441:3509], now have 1 active actors on pipe
2025-11-14T16:03:02.593453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.593461Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.593487Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|3cde2c7-42379d53-b92bc8f5-e581fe47_46 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.593495Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.593497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.593501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.593503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.593508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.593514Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.593516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.593518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.593528Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.593540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.593667Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|3cde2c7-42379d53-b92bc8f5-e581fe47_46"
    Status: Active
    SeqNo: 0
  }
}
2025-11-14T16:03:02.593894Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951441:3509] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|3cde2c7-42379d53-b92bc8f5-e581fe47_46"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 46
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.596089Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.596454Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951444:3510], now have 1 active actors on pipe
2025-11-14T16:03:02.596798Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.596810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.596826Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 46 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.596836Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.596846Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.596850Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.596858Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.596860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.596864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.596886Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 46 partNo 0
2025-11-14T16:03:02.596942Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 46 partNo 0 FormedBlobsCount 0 NewHead: Offset 46 PartNo 0 PackedSize 262214 count 1 nextOffset 47 batches 1
2025-11-14T16:03:02.596948Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.596950Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.596952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.597066Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 46,1 HeadOffset 46 endOffset 46 curOffset 47 d0000000000_00000000000000000046_00000_0000000001_00000? size 262204 WTime 1763136182596
2025-11-14T16:03:02.597106Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.597118Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 46 partNo 0 count 1 size 262204
2025-11-14T16:03:02.599447Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 46 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.599469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.599489Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.599500Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.599514Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 46, partNo: 0, Offset: 46 is stored on disk
2025-11-14T16:03:02.599559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.599565Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.599567Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.599570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.599572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.599578Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.599588Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.599643Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 46 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.599733Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 46
    Offset: 46
    WriteTimestampMS: 1763136182596
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-14T16:03:02.600083Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951444:3510] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.600670Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.601100Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951448:3512], now have 1 active actors on pipe
2025-11-14T16:03:02.601242Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.601257Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.601284Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|e246ff7e-c7f65642-cb546c80-e1b726e9_47 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.601295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.601298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.601302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.601304Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.601308Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.601316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.601318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.601320Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.601332Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.601347Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.601451Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|e246ff7e-c7f65642-cb546c80-e1b726e9_47"
    Status: Active
    SeqNo: 0
  }
}
2025-11-14T16:03:02.601683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951448:3512] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|e246ff7e-c7f65642-cb546c80-e1b726e9_47"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 47
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.603988Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.604316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951451:3513], now have 1 active actors on pipe
2025-11-14T16:03:02.604711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.604721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.604747Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 47 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.604756Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.604767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.604770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.604777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.604779Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.604783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.604805Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 47 partNo 0
2025-11-14T16:03:02.604834Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 47 partNo 0 FormedBlobsCount 0 NewHead: Offset 47 PartNo 0 PackedSize 262214 count 1 nextOffset 48 batches 1
2025-11-14T16:03:02.604842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.604844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.604847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.604983Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 47,1 HeadOffset 47 endOffset 47 curOffset 48 d0000000000_00000000000000000047_00000_0000000001_00000? size 262204 WTime 1763136182604
2025-11-14T16:03:02.605012Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.605022Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 47 partNo 0 count 1 size 262204
2025-11-14T16:03:02.607052Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 47 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.607077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.607096Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.607107Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.607109Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 47 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.607116Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 47, partNo: 0, Offset: 47 is stored on disk
2025-11-14T16:03:02.607145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.607148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.607151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.607153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.607155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.607162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.607172Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.607238Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.607462Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951451:3513] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 47
    Offset: 47
    WriteTimestampMS: 1763136182604
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.608067Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.608372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951455:3515], now have 1 active actors on pipe
2025-11-14T16:03:02.608447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.608456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.608486Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ba1131c5-71e7302b-8204a87c-382a76c9_48 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.608496Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.608499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.608503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.608505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.608509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.608515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.608520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.608523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.608532Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.608547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.608600Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ba1131c5-71e7302b-8204a87c-382a76c9_48"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
2025-11-14T16:03:02.608938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951455:3515] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ba1131c5-71e7302b-8204a87c-382a76c9_48"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 48
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.611069Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.611561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951458:3516], now have 1 active actors on pipe
2025-11-14T16:03:02.611711Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.611720Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.611737Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 48 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.611748Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.611758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.611765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.611773Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.611774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.611778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.611798Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 48 partNo 0
2025-11-14T16:03:02.611833Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 48 partNo 0 FormedBlobsCount 0 NewHead: Offset 48 PartNo 0 PackedSize 262214 count 1 nextOffset 49 batches 1
2025-11-14T16:03:02.611843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.611845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.611848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.611955Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 48,1 HeadOffset 48 endOffset 48 curOffset 49 d0000000000_00000000000000000048_00000_0000000001_00000? size 262204 WTime 1763136182611
2025-11-14T16:03:02.611991Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.612005Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 48 partNo 0 count 1 size 262204
2025-11-14T16:03:02.614306Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 48 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.614333Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.614352Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.614357Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 48 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.614359Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.614368Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 48, partNo: 0, Offset: 48 is stored on disk
2025-11-14T16:03:02.614398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.614407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.614410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.614412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.614414Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.614420Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.614428Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.614500Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.614770Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951458:3516] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 48
    Offset: 48
    WriteTimestampMS: 1763136182611
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.615373Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.615733Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951462:3518], now have 1 active actors on pipe
2025-11-14T16:03:02.615875Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.615885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.615933Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|be5a405d-2de7f9ba-471f7860-1f081dbe_49 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.615944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.615947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.615952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.615954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.615958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.615966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.615969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.615972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.615983Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.616000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.616052Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|be5a405d-2de7f9ba-471f7860-1f081dbe_49"
    Status: Active
    SeqNo: 0
  }
}
2025-11-14T16:03:02.616241Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951462:3518] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|be5a405d-2de7f9ba-471f7860-1f081dbe_49"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 49
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.618383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.618403Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.618406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.618412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.618429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.618615Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.619082Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951465:3519], now have 1 active actors on pipe
2025-11-14T16:03:02.619519Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.619540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.619571Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 49 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.619589Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.619609Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.619614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.619628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.619632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.619643Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.619680Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 49 partNo 0
2025-11-14T16:03:02.619736Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 49 partNo 0 FormedBlobsCount 0 NewHead: Offset 49 PartNo 0 PackedSize 262214 count 1 nextOffset 50 batches 1
2025-11-14T16:03:02.619749Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.619752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.619755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.619903Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 49,1 HeadOffset 49 endOffset 49 curOffset 50 d0000000000_00000000000000000049_00000_0000000001_00000? size 262204 WTime 1763136182619
2025-11-14T16:03:02.619952Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.619972Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 49 partNo 0 count 1 size 262204
2025-11-14T16:03:02.622543Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 49 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.622576Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.622592Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 49 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.622599Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.622608Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.622619Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 49, partNo: 0, Offset: 49 is stored on disk
2025-11-14T16:03:02.622657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.622664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.622666Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.622669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.622672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.622678Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.622690Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.622763Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.623123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951465:3519] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 49
    Offset: 49
    WriteTimestampMS: 1763136182619
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.623701Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.624060Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951468:3520], now have 1 active actors on pipe
2025-11-14T16:03:02.624201Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.624220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.624264Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|5a870d16-efa56ef8-6acd7cd5-f2f52397_50 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.624280Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.624288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.624296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.624302Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.624307Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.624316Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.624319Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.624322Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.624349Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.624367Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.624417Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|5a870d16-efa56ef8-6acd7cd5-f2f52397_50"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
2025-11-14T16:03:02.624645Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951468:3520] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|5a870d16-efa56ef8-6acd7cd5-f2f52397_50"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 50
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.626974Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.627628Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951471:3521], now have 1 active actors on pipe
2025-11-14T16:03:02.628099Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.628111Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.628130Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 50 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.628140Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.628157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.628164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.628174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.628180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.628186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.628212Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 50 partNo 0
2025-11-14T16:03:02.628247Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 50 partNo 0 FormedBlobsCount 0 NewHead: Offset 50 PartNo 0 PackedSize 262214 count 1 nextOffset 51 batches 1
2025-11-14T16:03:02.628271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.628274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.628276Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.628384Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 50,1 HeadOffset 50 endOffset 50 curOffset 51 d0000000000_00000000000000000050_00000_0000000001_00000? size 262204 WTime 1763136182628
2025-11-14T16:03:02.628423Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.628440Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 50 partNo 0 count 1 size 262204
2025-11-14T16:03:02.631153Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 50 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.631177Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.631201Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.631215Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.631224Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 50, partNo: 0, Offset: 50 is stored on disk
2025-11-14T16:03:02.631252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.631255Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.631257Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.631259Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.631260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.631268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.631278Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.631310Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 50 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.631384Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.631688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951471:3521] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 50
    Offset: 50
    WriteTimestampMS: 1763136182628
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.632267Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.632666Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951475:3523], now have 1 active actors on pipe
2025-11-14T16:03:02.632762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.632781Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.632818Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|c1fd279b-7c0d3100-2d2a0652-cc3b0f1c_51 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.632835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.632840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.632845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.632847Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.632852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.632862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.632864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.632867Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.632880Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.632896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.632964Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|c1fd279b-7c0d3100-2d2a0652-cc3b0f1c_51"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
2025-11-14T16:03:02.633295Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951475:3523] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|c1fd279b-7c0d3100-2d2a0652-cc3b0f1c_51"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 51
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.635266Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.635650Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951478:3524], now have 1 active actors on pipe
2025-11-14T16:03:02.636077Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.636092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.636108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 51 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.636117Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.636136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.636143Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.636152Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.636159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.636175Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.636201Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 51 partNo 0
2025-11-14T16:03:02.636235Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 51 partNo 0 FormedBlobsCount 0 NewHead: Offset 51 PartNo 0 PackedSize 262214 count 1 nextOffset 52 batches 1
2025-11-14T16:03:02.636241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.636244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.636246Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.636358Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 51,1 HeadOffset 51 endOffset 51 curOffset 52 d0000000000_00000000000000000051_00000_0000000001_00000? size 262204 WTime 1763136182633
2025-11-14T16:03:02.636398Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.636415Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 51 partNo 0 count 1 size 262204
2025-11-14T16:03:02.638461Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 51 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.638487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.638502Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 51 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.638507Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.638515Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.638524Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 51, partNo: 0, Offset: 51 is stored on disk
2025-11-14T16:03:02.638550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.638557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.638559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.638562Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.638564Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.638571Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.638580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.638654Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.639143Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951478:3524] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 51
    Offset: 51
    WriteTimestampMS: 1763136182633
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.642110Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.642546Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951485:3527], now have 1 active actors on pipe
2025-11-14T16:03:02.642567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.642573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.642601Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|7ee9753e-fbdc5820-f2b0247f-2cb4dd05_52 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.642614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.642620Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.642625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.642627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.642634Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.642642Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.642648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.642650Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.642661Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.642672Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.642746Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.642964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951485:3527] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|7ee9753e-fbdc5820-f2b0247f-2cb4dd05_52"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|7ee9753e-fbdc5820-f2b0247f-2cb4dd05_52"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 52
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.645689Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.648691Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951488:3529], now have 1 active actors on pipe
2025-11-14T16:03:02.648807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.648820Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.648841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 52 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.648856Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.648874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.648882Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.648893Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.648917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.648926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.648956Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 52 partNo 0
2025-11-14T16:03:02.648998Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 52 partNo 0 FormedBlobsCount 0 NewHead: Offset 52 PartNo 0 PackedSize 262214 count 1 nextOffset 53 batches 1
2025-11-14T16:03:02.649007Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.649011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.649014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.649129Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 52,1 HeadOffset 52 endOffset 52 curOffset 53 d0000000000_00000000000000000052_00000_0000000001_00000? size 262204 WTime 1763136182648
2025-11-14T16:03:02.649181Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.649194Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 52 partNo 0 count 1 size 262204
2025-11-14T16:03:02.654518Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 52 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.654545Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.654563Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.654572Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.654587Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 52, partNo: 0, Offset: 52 is stored on disk
2025-11-14T16:03:02.654626Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.654633Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.654635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.654637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.654639Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.654645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.654656Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.654707Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 52 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.654762Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.655085Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951488:3529] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 52
    Offset: 52
    WriteTimestampMS: 1763136182648
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.655796Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.656534Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951492:3531], now have 1 active actors on pipe
2025-11-14T16:03:02.656559Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.656565Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.656590Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|36626300-73b41cb-a5fa9088-7f6a756f_53 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.656605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.656608Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.656611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.656613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.656616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.656623Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.656625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.656628Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.656637Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.656647Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.656742Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|36626300-73b41cb-a5fa9088-7f6a756f_53"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|36626300-73b41cb-a5fa9088-7f6a756f_53"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 53
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.659124Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.660303Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951492:3531] destroyed
2025-11-14T16:03:02.660326Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951495:3532], now have 1 active actors on pipe
2025-11-14T16:03:02.660436Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.660448Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.660470Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 53 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.660490Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.660505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.660513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.660521Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.660527Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.660531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.660560Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 53 partNo 0
2025-11-14T16:03:02.660595Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 53 partNo 0 FormedBlobsCount 0 NewHead: Offset 53 PartNo 0 PackedSize 262214 count 1 nextOffset 54 batches 1
2025-11-14T16:03:02.660610Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.660613Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.660616Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.660729Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 53,1 HeadOffset 53 endOffset 53 curOffset 54 d0000000000_00000000000000000053_00000_0000000001_00000? size 262204 WTime 1763136182660
2025-11-14T16:03:02.660768Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.660783Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 53 partNo 0 count 1 size 262204
2025-11-14T16:03:02.666665Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 53 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.666697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.666718Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.666728Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.666746Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 53, partNo: 0, Offset: 53 is stored on disk
2025-11-14T16:03:02.666784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.666788Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 53 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.666791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.666794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.666797Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.666799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.666806Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.666818Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.666941Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 53
    Offset: 53
    WriteTimestampMS: 1763136182660
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 6
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.667535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951495:3532] destroyed
2025-11-14T16:03:02.667985Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.668768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951499:3534], now have 1 active actors on pipe
2025-11-14T16:03:02.668927Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.668938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.668972Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|1944e9a5-c22f15c6-8dec1a18-c7edef4c_54 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.668981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.668984Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.668988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.668991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.668995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.669006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.669012Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.669015Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.669029Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.669039Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.669093Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.669403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951499:3534] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|1944e9a5-c22f15c6-8dec1a18-c7edef4c_54"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|1944e9a5-c22f15c6-8dec1a18-c7edef4c_54"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 54
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.672012Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.672484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951502:3535], now have 1 active actors on pipe
2025-11-14T16:03:02.672888Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.672897Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.672935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 54 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.672945Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.672958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.672961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.672970Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.672972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.672976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.673000Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 54 partNo 0
2025-11-14T16:03:02.673030Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 54 partNo 0 FormedBlobsCount 0 NewHead: Offset 54 PartNo 0 PackedSize 262214 count 1 nextOffset 55 batches 1
2025-11-14T16:03:02.673037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.673041Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.673122Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.673242Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 54,1 HeadOffset 54 endOffset 54 curOffset 55 d0000000000_00000000000000000054_00000_0000000001_00000? size 262204 WTime 1763136182673
2025-11-14T16:03:02.673275Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.673285Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 54 partNo 0 count 1 size 262204
2025-11-14T16:03:02.676845Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 54 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.676873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.676894Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.676916Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 54 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.676925Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.676935Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 54, partNo: 0, Offset: 54 is stored on disk
2025-11-14T16:03:02.676965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.676972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.676974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.676976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.676978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.676987Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.676998Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.677081Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.677411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951502:3535] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 54
    Offset: 54
    WriteTimestampMS: 1763136182672
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.678091Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.678485Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951506:3537], now have 1 active actors on pipe
2025-11-14T16:03:02.678580Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.678589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.678616Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9c3b7f04-baae0ae2-4f01443f-58212ef1_55 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.678622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.678625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.678630Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.678631Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.678635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.678644Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.678647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.678649Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.678657Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.678665Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.678770Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9c3b7f04-baae0ae2-4f01443f-58212ef1_55"
    Status: Active
    SeqNo: 0
  }
}
2025-11-14T16:03:02.679004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951506:3537] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9c3b7f04-baae0ae2-4f01443f-58212ef1_55"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 55
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.681111Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.681491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951509:3538], now have 1 active actors on pipe
2025-11-14T16:03:02.681827Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.681835Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.681854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 55 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.681861Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.681872Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.681874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.681881Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.681883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.681886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.681907Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 55 partNo 0
2025-11-14T16:03:02.681935Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 55 partNo 0 FormedBlobsCount 0 NewHead: Offset 55 PartNo 0 PackedSize 262214 count 1 nextOffset 56 batches 1
2025-11-14T16:03:02.681941Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.681942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.681946Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.682045Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 55,1 HeadOffset 55 endOffset 55 curOffset 56 d0000000000_00000000000000000055_00000_0000000001_00000? size 262204 WTime 1763136182681
2025-11-14T16:03:02.682074Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.682084Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 55 partNo 0 count 1 size 262204
2025-11-14T16:03:02.684709Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 55 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.684731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.684748Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.684755Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.684764Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 55, partNo: 0, Offset: 55 is stored on disk
2025-11-14T16:03:02.684788Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.684791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.684794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.684796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.684798Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.684803Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.684811Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.684832Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 55 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.684974Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 55
    Offset: 55
    WriteTimestampMS: 1763136182681
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
2025-11-14T16:03:02.685338Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951509:3538] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.685910Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.686183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951514:3541], now have 1 active actors on pipe
2025-11-14T16:03:02.686299Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.686306Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.686328Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|99047fb1-f23ba8e0-221954a0-ff60869b_56 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.686335Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.686338Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.686341Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.686344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.686346Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.686353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.686355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.686360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.686367Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.686375Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.686475Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.686724Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951514:3541] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|99047fb1-f23ba8e0-221954a0-ff60869b_56"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|99047fb1-f23ba8e0-221954a0-ff60869b_56"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 56
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.689174Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.689524Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951517:3542], now have 1 active actors on pipe
2025-11-14T16:03:02.689845Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.689854Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.689869Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 56 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.689877Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.689887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.689890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.689898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.689900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.689904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.689926Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 56 partNo 0
2025-11-14T16:03:02.689953Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 56 partNo 0 FormedBlobsCount 0 NewHead: Offset 56 PartNo 0 PackedSize 262214 count 1 nextOffset 57 batches 1
2025-11-14T16:03:02.689959Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.689961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.689965Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.690066Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 56,1 HeadOffset 56 endOffset 56 curOffset 57 d0000000000_00000000000000000056_00000_0000000001_00000? size 262204 WTime 1763136182689
2025-11-14T16:03:02.690095Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.690106Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 56 partNo 0 count 1 size 262204
2025-11-14T16:03:02.691961Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 56 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.691982Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.691997Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.692000Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 56 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.692004Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.692013Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 56, partNo: 0, Offset: 56 is stored on disk
2025-11-14T16:03:02.692039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.692042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.692044Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.692048Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.692050Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.692056Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.692064Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.692133Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.692373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951517:3542] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 56
    Offset: 56
    WriteTimestampMS: 1763136182689
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.692966Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.693252Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951521:3544], now have 1 active actors on pipe
2025-11-14T16:03:02.693371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.693379Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.693404Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|594dfe1c-9d78f8c2-b97a1016-629e63ad_57 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.693410Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.693412Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.693415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.693416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.693418Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.693425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.693427Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.693429Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.693437Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.693445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.693549Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|594dfe1c-9d78f8c2-b97a1016-629e63ad_57"
    Status: Active
    SeqNo: 0
  }
}
2025-11-14T16:03:02.693758Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951521:3544] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|594dfe1c-9d78f8c2-b97a1016-629e63ad_57"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 57
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.695766Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.696442Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951524:3545], now have 1 active actors on pipe
2025-11-14T16:03:02.696543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.696549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.696567Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 57 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.696578Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.696594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.696596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.696604Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.696606Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.696609Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.696631Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 57 partNo 0
2025-11-14T16:03:02.696657Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 57 partNo 0 FormedBlobsCount 0 NewHead: Offset 57 PartNo 0 PackedSize 262214 count 1 nextOffset 58 batches 1
2025-11-14T16:03:02.696662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.696664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.696667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.696772Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 57,1 HeadOffset 57 endOffset 57 curOffset 58 d0000000000_00000000000000000057_00000_0000000001_00000? size 262204 WTime 1763136182696
2025-11-14T16:03:02.696803Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.696814Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 57 partNo 0 count 1 size 262204
2025-11-14T16:03:02.698821Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 57 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.698840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.698853Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 57 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.698856Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.698863Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.698870Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 57, partNo: 0, Offset: 57 is stored on disk
2025-11-14T16:03:02.698896Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.698898Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.698900Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.698902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.698904Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.698910Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.698918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.699036Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 57
    Offset: 57
    WriteTimestampMS: 1763136182696
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-14T16:03:02.699315Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951524:3545] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.699751Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.700024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951528:3547], now have 1 active actors on pipe
2025-11-14T16:03:02.700175Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.700188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.700214Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|2951ff48-4ccbe996-d78e85e0-b2083c38_58 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.700226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.700233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.700238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.700240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.700244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.700250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.700258Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.700260Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.700269Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.700285Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|2951ff48-4ccbe996-d78e85e0-b2083c38_58"
    Status: Active
    SeqNo: 0
  }
}
2025-11-14T16:03:02.700363Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC request to localhost:2468
2025-11-14T16:03:02.700600Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951528:3547] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|2951ff48-4ccbe996-d78e85e0-b2083c38_58"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 58
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.702506Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.702837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951531:3548], now have 1 active actors on pipe
2025-11-14T16:03:02.703228Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.703240Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.703258Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 58 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.703271Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.703287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.703294Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.703303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.703309Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.703313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.703334Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 58 partNo 0
2025-11-14T16:03:02.703366Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 58 partNo 0 FormedBlobsCount 0 NewHead: Offset 58 PartNo 0 PackedSize 262214 count 1 nextOffset 59 batches 1
2025-11-14T16:03:02.703377Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.703380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.703382Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.703502Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 58,1 HeadOffset 58 endOffset 58 curOffset 59 d0000000000_00000000000000000058_00000_0000000001_00000? size 262204 WTime 1763136182703
2025-11-14T16:03:02.703536Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.703550Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 58 partNo 0 count 1 size 262204
2025-11-14T16:03:02.705883Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 58 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.705907Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.705922Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.705933Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.705941Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 58, partNo: 0, Offset: 58 is stored on disk
2025-11-14T16:03:02.705979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.705985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.705987Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.705990Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.705992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.705998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.706006Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.706065Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 58 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.706111Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 58
    Offset: 58
    WriteTimestampMS: 1763136182703
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-14T16:03:02.706380Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951531:3548] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.706858Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.710611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951535:3550], now have 1 active actors on pipe
2025-11-14T16:03:02.710752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.710763Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.710795Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|99940876-d999aa0-6826531f-bb825c7e_59 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.710810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.710813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.710818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.710825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.710829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.710839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.710845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.710849Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.710863Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.710879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.710958Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.711223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951535:3550] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|99940876-d999aa0-6826531f-bb825c7e_59"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|99940876-d999aa0-6826531f-bb825c7e_59"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 59
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.714041Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.714397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951538:3551], now have 1 active actors on pipe
2025-11-14T16:03:02.714713Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.714726Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.714749Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 59 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.714764Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.714781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.714789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.714799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.714814Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.714824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.714851Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 59 partNo 0
2025-11-14T16:03:02.714890Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 59 partNo 0 FormedBlobsCount 0 NewHead: Offset 59 PartNo 0 PackedSize 262214 count 1 nextOffset 60 batches 1
2025-11-14T16:03:02.714902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.714908Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.714912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.715043Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 59,1 HeadOffset 59 endOffset 59 curOffset 60 d0000000000_00000000000000000059_00000_0000000001_00000? size 262204 WTime 1763136182715
2025-11-14T16:03:02.715085Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.715101Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 59 partNo 0 count 1 size 262204
2025-11-14T16:03:02.717087Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 59 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.717115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.717124Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 59 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.717135Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.717144Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.717154Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 59, partNo: 0, Offset: 59 is stored on disk
2025-11-14T16:03:02.717203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.717211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.717213Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.717215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.717217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.717224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.717234Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.717307Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.717572Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951538:3551] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 59
    Offset: 59
    WriteTimestampMS: 1763136182714
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.718219Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.718626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951542:3553], now have 1 active actors on pipe
2025-11-14T16:03:02.718643Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.718648Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.718662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.718669Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.718672Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.718674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.718676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.718701Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|dc5d7871-3be7a2b4-c1cf9575-5198f030_60 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.718709Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.718712Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.718716Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.718718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.718721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.718727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.718732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.718735Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.718741Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.718748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.718817Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.719024Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951542:3553] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|dc5d7871-3be7a2b4-c1cf9575-5198f030_60"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|dc5d7871-3be7a2b4-c1cf9575-5198f030_60"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 60
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.721093Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.721561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951545:3554], now have 1 active actors on pipe
2025-11-14T16:03:02.721952Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.721967Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.722001Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 60 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.722012Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.722025Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.722033Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.722040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.722042Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.722046Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.722068Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 60 partNo 0
2025-11-14T16:03:02.722107Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 60 partNo 0 FormedBlobsCount 0 NewHead: Offset 60 PartNo 0 PackedSize 262214 count 1 nextOffset 61 batches 1
2025-11-14T16:03:02.722117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.722120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.722123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.722233Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 60,1 HeadOffset 60 endOffset 60 curOffset 61 d0000000000_00000000000000000060_00000_0000000001_00000? size 262204 WTime 1763136182722
2025-11-14T16:03:02.722286Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.722303Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 60 partNo 0 count 1 size 262204
2025-11-14T16:03:02.724572Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 60 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.724596Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.724614Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 60 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.724616Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.724625Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.724638Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 60, partNo: 0, Offset: 60 is stored on disk
2025-11-14T16:03:02.724674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.724681Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.724684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.724688Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.724690Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.724697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.724707Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.724793Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 60
    Offset: 60
    WriteTimestampMS: 1763136182721
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-14T16:03:02.725106Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951545:3554] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.725613Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.725989Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951549:3556], now have 1 active actors on pipe
2025-11-14T16:03:02.726148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.726159Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.726182Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a9ba925d-db17d5a8-58a32a2f-8f031470_61 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.726192Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.726194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.726199Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.726206Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.726209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.726217Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.726225Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.726229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.726237Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.726248Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.726309Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.726547Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951549:3556] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a9ba925d-db17d5a8-58a32a2f-8f031470_61"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a9ba925d-db17d5a8-58a32a2f-8f031470_61"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 61
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.728651Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.729019Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951552:3557], now have 1 active actors on pipe
2025-11-14T16:03:02.729355Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.729370Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.729393Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 61 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.729407Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.729422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.729428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.729436Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.729438Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.729441Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.729463Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 61 partNo 0
2025-11-14T16:03:02.729501Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 61 partNo 0 FormedBlobsCount 0 NewHead: Offset 61 PartNo 0 PackedSize 262214 count 1 nextOffset 62 batches 1
2025-11-14T16:03:02.729510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.729512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.729515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.729625Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 61,1 HeadOffset 61 endOffset 61 curOffset 62 d0000000000_00000000000000000061_00000_0000000001_00000? size 262204 WTime 1763136182729
2025-11-14T16:03:02.729655Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.729663Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 61 partNo 0 count 1 size 262204
2025-11-14T16:03:02.731953Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 61 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.731980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.731999Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.731998Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 61 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.732008Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.732017Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 61, partNo: 0, Offset: 61 is stored on disk
2025-11-14T16:03:02.732047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.732055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.732057Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.732060Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.732062Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.732069Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.732080Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.732166Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.732466Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951552:3557] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 61
    Offset: 61
    WriteTimestampMS: 1763136182729
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.733242Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.733615Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951556:3559], now have 1 active actors on pipe
2025-11-14T16:03:02.733683Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.733694Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.733724Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|16aa66de-3fa11c1e-1d1b4e45-b0293f5f_62 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.733732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.733734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.733738Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.733740Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.733743Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.733750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.733753Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.733755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.733765Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.733776Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.733836Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|16aa66de-3fa11c1e-1d1b4e45-b0293f5f_62"
    Status: Active
    SeqNo: 0
  }
}
2025-11-14T16:03:02.734059Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951556:3559] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|16aa66de-3fa11c1e-1d1b4e45-b0293f5f_62"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 62
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.736602Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.736942Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951559:3560], now have 1 active actors on pipe
2025-11-14T16:03:02.737441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.737455Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.737474Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 62 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.737493Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.737507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.737510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.737517Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.737519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.737523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.737547Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 62 partNo 0
2025-11-14T16:03:02.737582Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 62 partNo 0 FormedBlobsCount 0 NewHead: Offset 62 PartNo 0 PackedSize 262214 count 1 nextOffset 63 batches 1
2025-11-14T16:03:02.737592Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.737594Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.737597Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.738006Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 62,1 HeadOffset 62 endOffset 62 curOffset 63 d0000000000_00000000000000000062_00000_0000000001_00000? size 262204 WTime 1763136182737
2025-11-14T16:03:02.738043Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.738058Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 62 partNo 0 count 1 size 262204
2025-11-14T16:03:02.740353Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 62 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.740384Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.740394Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.740406Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.740414Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.740423Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 62, partNo: 0, Offset: 62 is stored on disk
2025-11-14T16:03:02.740454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.740464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.740466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.740468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.740470Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.740479Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.740495Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.740553Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.740809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951559:3560] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 62
    Offset: 62
    WriteTimestampMS: 1763136182737
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.741513Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.741867Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951563:3562], now have 1 active actors on pipe
2025-11-14T16:03:02.741991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.742005Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.742041Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b3acfacb-af09e0b9-fcc1d044-6d0e4714_63 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.742052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.742055Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.742059Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.742061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.742064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.742072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.742077Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.742079Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.742090Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.742102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.742189Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b3acfacb-af09e0b9-fcc1d044-6d0e4714_63"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
2025-11-14T16:03:02.742550Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951563:3562] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b3acfacb-af09e0b9-fcc1d044-6d0e4714_63"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 63
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.744684Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.745025Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951566:3563], now have 1 active actors on pipe
2025-11-14T16:03:02.745397Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.745411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.745431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 63 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.745444Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.745461Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.745469Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.745478Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.745484Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.745489Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.745515Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 63 partNo 0
2025-11-14T16:03:02.745552Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 63 partNo 0 FormedBlobsCount 0 NewHead: Offset 63 PartNo 0 PackedSize 262214 count 1 nextOffset 64 batches 1
2025-11-14T16:03:02.745563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.745568Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.745570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.745677Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 63,1 HeadOffset 63 endOffset 63 curOffset 64 d0000000000_00000000000000000063_00000_0000000001_00000? size 262204 WTime 1763136182745
2025-11-14T16:03:02.745716Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.745732Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 63 partNo 0 count 1 size 262204
2025-11-14T16:03:02.748130Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 63 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.748160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.748171Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 63 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.748181Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.748188Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.748196Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 63, partNo: 0, Offset: 63 is stored on disk
2025-11-14T16:03:02.748228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.748238Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.748241Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.748244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.748245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.748253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000032_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748257Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000033_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748261Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000034_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748268Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000035_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748271Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000036_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748274Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000037_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000038_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748281Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000039_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748284Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000040_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748287Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000041_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748290Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000042_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748292Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000043_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748296Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000044_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000045_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748301Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000046_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000047_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000048_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000049_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000050_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000051_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748319Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000052_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748322Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000053_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748325Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000054_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748328Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000055_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748330Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000056_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000057_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748335Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000058_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748338Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000059_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748340Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000060_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000061_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748345Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000062_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748347Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000063_00000_0000000001_00000? (262204)
2025-11-14T16:03:02.748349Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 32 keys were taken away. Let's read 8390528 bytes
2025-11-14T16:03:02.748360Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.748374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 32 blobs
2025-11-14T16:03:02.748377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000032_00000_0000000001_00000?
2025-11-14T16:03:02.748388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000033_00000_0000000001_00000?
2025-11-14T16:03:02.748400Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000034_00000_0000000001_00000?
2025-11-14T16:03:02.748417Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000035_00000_0000000001_00000?
2025-11-14T16:03:02.748430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000036_00000_0000000001_00000?
2025-11-14T16:03:02.748448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000037_00000_0000000001_00000?
2025-11-14T16:03:02.748456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000038_00000_0000000001_00000?
2025-11-14T16:03:02.748434Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.748465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000039_00000_0000000001_00000?
2025-11-14T16:03:02.748476Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000040_00000_0000000001_00000?
2025-11-14T16:03:02.748490Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000041_00000_0000000001_00000?
2025-11-14T16:03:02.748494Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000042_00000_0000000001_00000?
2025-11-14T16:03:02.748498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000043_00000_0000000001_00000?
2025-11-14T16:03:02.748505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000044_00000_0000000001_00000?
2025-11-14T16:03:02.748509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000045_00000_0000000001_00000?
2025-11-14T16:03:02.748513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000046_00000_0000000001_00000?
2025-11-14T16:03:02.748521Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000047_00000_0000000001_00000?
2025-11-14T16:03:02.748532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000048_00000_0000000001_00000?
2025-11-14T16:03:02.748576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000049_00000_0000000001_00000?
2025-11-14T16:03:02.748589Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000050_00000_0000000001_00000?
2025-11-14T16:03:02.748596Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000051_00000_0000000001_00000?
2025-11-14T16:03:02.748603Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000052_00000_0000000001_00000?
2025-11-14T16:03:02.748610Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000053_00000_0000000001_00000?
2025-11-14T16:03:02.748620Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000054_00000_0000000001_00000?
2025-11-14T16:03:02.748628Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000055_00000_0000000001_00000?
2025-11-14T16:03:02.748637Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000056_00000_0000000001_00000?
2025-11-14T16:03:02.748649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000057_00000_0000000001_00000?
2025-11-14T16:03:02.748655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000058_00000_0000000001_00000?
2025-11-14T16:03:02.748662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000059_00000_0000000001_00000?
2025-11-14T16:03:02.748670Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000060_00000_0000000001_00000?
2025-11-14T16:03:02.748679Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000061_00000_0000000001_00000?
2025-11-14T16:03:02.748688Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000062_00000_0000000001_00000?
CallPersQueueGRPC response:
2025-11-14T16:03:02.748696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000063_00000_0000000001_00000?
2025-11-14T16:03:02.748710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 32 blobs for compaction
2025-11-14T16:03:02.748721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951566:3563] destroyed
2025-11-14T16:03:02.748729Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 32 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748735Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 33 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748740Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 34 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748742Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 35 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 63
    Offset: 63
    WriteTimestampMS: 1763136182745
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-14T16:03:02.748745Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 36 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748748Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 37 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748752Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 38 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748755Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 39 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748758Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 40 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748761Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 41 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748765Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 42 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748768Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 43 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748771Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 44 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748774Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 45 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748778Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 46 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748782Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 47 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748785Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 48 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748788Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 49 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748792Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 50 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748795Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 51 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748798Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 52 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748801Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 53 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
CallPersQueueGRPC request to localhost:2468
2025-11-14T16:03:02.748804Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 54 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748807Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 55 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748811Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 56 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.748814Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 57 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748817Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 58 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748820Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 59 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748823Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 60 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748826Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 61 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748828Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 62 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748831Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 63 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:02.748846Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 32 blobs are from cache.
2025-11-14T16:03:02.748860Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 32 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748864Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 33 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748867Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 34 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748870Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 35 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748873Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 36 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748876Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 37 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748877Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-14T16:03:02.748879Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 38 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748883Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 39 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/32] d0000000000_00000000000000000032_00000_0000000001_00000?
2025-11-14T16:03:02.748886Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 40 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.748889Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 41 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000032_00000_0000000001_00000?, parameters.CurOffset=32
2025-11-14T16:03:02.748892Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 42 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748894Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 43 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748897Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 44 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748917Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 45 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748920Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 46 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748923Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 47 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748926Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 48 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748930Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 49 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748933Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 50 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748935Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 51 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748938Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 52 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748941Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 53 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748944Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 54 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748947Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 55 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748950Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 56 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748952Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 57 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748954Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 58 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748958Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 59 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748961Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 60 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748966Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 61 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748969Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.748972Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 63 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:02.749002Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 32.0/1
2025-11-14T16:03:02.749009Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 32 partNo 0
2025-11-14T16:03:02.749032Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 32 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 262214 count 1 nextOffset 33 batches 1
2025-11-14T16:03:02.749040Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 32:0 appended
2025-11-14T16:03:02.749044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/32] d0000000000_00000000000000000033_00000_0000000001_00000?
2025-11-14T16:03:02.749046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.749049Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000033_00000_0000000001_00000?, parameters.CurOffset=33
2025-11-14T16:03:02.749135Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 33.0/1
2025-11-14T16:03:02.749145Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 33 partNo 0
2025-11-14T16:03:02.749391Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.749597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 33 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 524387 count 2 nextOffset 34 batches 1
2025-11-14T16:03:02.749607Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 33:0 appended
2025-11-14T16:03:02.749613Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/32] d0000000000_00000000000000000034_00000_0000000001_00000?
2025-11-14T16:03:02.749615Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.749617Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000034_00000_0000000001_00000?, parameters.CurOffset=34
2025-11-14T16:03:02.749686Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 34.0/1
2025-11-14T16:03:02.749696Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 34 partNo 0
2025-11-14T16:03:02.749709Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 34 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 786601 count 3 nextOffset 35 batches 2
2025-11-14T16:03:02.749711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 34:0 appended
2025-11-14T16:03:02.749714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/32] d0000000000_00000000000000000035_00000_0000000001_00000?
2025-11-14T16:03:02.749715Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.749717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000035_00000_0000000001_00000?, parameters.CurOffset=35
2025-11-14T16:03:02.749769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 35.0/1
2025-11-14T16:03:02.749777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 35 partNo 0
2025-11-14T16:03:02.750552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 35 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 1048774 count 4 nextOffset 36 batches 2
2025-11-14T16:03:02.750563Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 35:0 appended
2025-11-14T16:03:02.750568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/32] d0000000000_00000000000000000036_00000_0000000001_00000?
2025-11-14T16:03:02.750569Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.750571Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000036_00000_0000000001_00000?, parameters.CurOffset=36
2025-11-14T16:03:02.750626Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 36.0/1
2025-11-14T16:03:02.750633Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 36 partNo 0
2025-11-14T16:03:02.750643Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 36 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 1310988 count 5 nextOffset 37 batches 3
2025-11-14T16:03:02.750644Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 36:0 appended
2025-11-14T16:03:02.750647Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/32] d0000000000_00000000000000000037_00000_0000000001_00000?
2025-11-14T16:03:02.750648Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.750649Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000037_00000_0000000001_00000?, parameters.CurOffset=37
2025-11-14T16:03:02.750702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 37.0/1
2025-11-14T16:03:02.750710Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 37 partNo 0
2025-11-14T16:03:02.751435Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 37 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 1573161 count 6 nextOffset 38 batches 3
2025-11-14T16:03:02.751446Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 37:0 appended
2025-11-14T16:03:02.751449Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/32] d0000000000_00000000000000000038_00000_0000000001_00000?
2025-11-14T16:03:02.751450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.751452Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000038_00000_0000000001_00000?, parameters.CurOffset=38
2025-11-14T16:03:02.751505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 38.0/1
2025-11-14T16:03:02.751513Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 38 partNo 0
2025-11-14T16:03:02.751522Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 38 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 1835375 count 7 nextOffset 39 batches 4
2025-11-14T16:03:02.751524Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 38:0 appended
2025-11-14T16:03:02.751527Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/32] d0000000000_00000000000000000039_00000_0000000001_00000?
2025-11-14T16:03:02.751529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.751531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000039_00000_0000000001_00000?, parameters.CurOffset=39
2025-11-14T16:03:02.751591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 39.0/1
2025-11-14T16:03:02.751598Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 39 partNo 0
2025-11-14T16:03:02.752194Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 39 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 2097548 count 8 nextOffset 40 batches 4
2025-11-14T16:03:02.752204Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 39:0 appended
2025-11-14T16:03:02.752210Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[8/32] d0000000000_00000000000000000040_00000_0000000001_00000?
2025-11-14T16:03:02.752212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.752216Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000040_00000_0000000001_00000?, parameters.CurOffset=40
2025-11-14T16:03:02.752277Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 40.0/1
2025-11-14T16:03:02.752286Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 40 partNo 0
2025-11-14T16:03:02.752298Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 40 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 2359762 count 9 nextOffset 41 batches 5
2025-11-14T16:03:02.752305Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 40:0 appended
2025-11-14T16:03:02.752309Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[9/32] d0000000000_00000000000000000041_00000_0000000001_00000?
2025-11-14T16:03:02.752311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.752314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000041_00000_0000000001_00000?, parameters.CurOffset=41
2025-11-14T16:03:02.752364Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 41.0/1
2025-11-14T16:03:02.752370Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 41 partNo 0
2025-11-14T16:03:02.752993Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 41 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 2621935 count 10 nextOffset 42 batches 5
2025-11-14T16:03:02.753004Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 41:0 appended
2025-11-14T16:03:02.753010Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[10/32] d0000000000_00000000000000000042_00000_0000000001_00000?
2025-11-14T16:03:02.753012Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.753015Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000042_00000_0000000001_00000?, parameters.CurOffset=42
2025-11-14T16:03:02.753078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 42.0/1
2025-11-14T16:03:02.753083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 42 partNo 0
2025-11-14T16:03:02.753096Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 42 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 2884149 count 11 nextOffset 43 batches 6
2025-11-14T16:03:02.753102Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 42:0 appended
2025-11-14T16:03:02.753106Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[11/32] d0000000000_00000000000000000043_00000_0000000001_00000?
2025-11-14T16:03:02.753108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.753112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000043_00000_0000000001_00000?, parameters.CurOffset=43
2025-11-14T16:03:02.753174Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 43.0/1
2025-11-14T16:03:02.753183Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 43 partNo 0
2025-11-14T16:03:02.753821Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 43 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 3146322 count 12 nextOffset 44 batches 6
2025-11-14T16:03:02.753831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 43:0 appended
2025-11-14T16:03:02.753836Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[12/32] d0000000000_00000000000000000044_00000_0000000001_00000?
2025-11-14T16:03:02.753840Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.753843Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000044_00000_0000000001_00000?, parameters.CurOffset=44
2025-11-14T16:03:02.753901Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 44.0/1
2025-11-14T16:03:02.753911Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 44 partNo 0
2025-11-14T16:03:02.753922Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 44 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 3408536 count 13 nextOffset 45 batches 7
2025-11-14T16:03:02.753928Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 44:0 appended
2025-11-14T16:03:02.753932Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[13/32] d0000000000_00000000000000000045_00000_0000000001_00000?
2025-11-14T16:03:02.753934Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.753936Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000045_00000_0000000001_00000?, parameters.CurOffset=45
2025-11-14T16:03:02.753996Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 45.0/1
2025-11-14T16:03:02.754007Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 45 partNo 0
2025-11-14T16:03:02.754342Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 45 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 3670709 count 14 nextOffset 46 batches 7
2025-11-14T16:03:02.754352Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 45:0 appended
2025-11-14T16:03:02.754357Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[14/32] d0000000000_00000000000000000046_00000_0000000001_00000?
2025-11-14T16:03:02.754359Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.754362Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000046_00000_0000000001_00000?, parameters.CurOffset=46
2025-11-14T16:03:02.754423Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 46.0/1
2025-11-14T16:03:02.754432Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 46 partNo 0
2025-11-14T16:03:02.754456Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 46 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 3932923 count 15 nextOffset 47 batches 8
2025-11-14T16:03:02.754462Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 46:0 appended
2025-11-14T16:03:02.754467Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[15/32] d0000000000_00000000000000000047_00000_0000000001_00000?
2025-11-14T16:03:02.754469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.754473Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000047_00000_0000000001_00000?, parameters.CurOffset=47
2025-11-14T16:03:02.754525Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 47.0/1
2025-11-14T16:03:02.754532Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 47 partNo 0
2025-11-14T16:03:02.754871Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 47 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 4195096 count 16 nextOffset 48 batches 8
2025-11-14T16:03:02.754883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 47:0 appended
2025-11-14T16:03:02.754887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[16/32] d0000000000_00000000000000000048_00000_0000000001_00000?
2025-11-14T16:03:02.754889Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.754892Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000048_00000_0000000001_00000?, parameters.CurOffset=48
2025-11-14T16:03:02.754951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 48.0/1
2025-11-14T16:03:02.754959Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 48 partNo 0
2025-11-14T16:03:02.754970Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 48 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 4457310 count 17 nextOffset 49 batches 9
2025-11-14T16:03:02.754976Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 48:0 appended
2025-11-14T16:03:02.754979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[17/32] d0000000000_00000000000000000049_00000_0000000001_00000?
2025-11-14T16:03:02.754981Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.754984Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000049_00000_0000000001_00000?, parameters.CurOffset=49
2025-11-14T16:03:02.755036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 49.0/1
2025-11-14T16:03:02.755043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 49 partNo 0
2025-11-14T16:03:02.755729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 49 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 4719483 count 18 nextOffset 50 batches 9
2025-11-14T16:03:02.755744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 49:0 appended
2025-11-14T16:03:02.755749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[18/32] d0000000000_00000000000000000050_00000_0000000001_00000?
2025-11-14T16:03:02.755751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.755754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000050_00000_0000000001_00000?, parameters.CurOffset=50
2025-11-14T16:03:02.755811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 50.0/1
2025-11-14T16:03:02.755818Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 50 partNo 0
2025-11-14T16:03:02.755826Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 50 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 4981697 count 19 nextOffset 51 batches 10
2025-11-14T16:03:02.755828Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 50:0 appended
2025-11-14T16:03:02.755830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[19/32] d0000000000_00000000000000000051_00000_0000000001_00000?
2025-11-14T16:03:02.755831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.755833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000051_00000_0000000001_00000?, parameters.CurOffset=51
2025-11-14T16:03:02.755880Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 51.0/1
2025-11-14T16:03:02.755887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 51 partNo 0
2025-11-14T16:03:02.756457Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 51 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 5243870 count 20 nextOffset 52 batches 10
2025-11-14T16:03:02.756465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 51:0 appended
2025-11-14T16:03:02.756469Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[20/32] d0000000000_00000000000000000052_00000_0000000001_00000?
2025-11-14T16:03:02.756470Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.756472Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000052_00000_0000000001_00000?, parameters.CurOffset=52
2025-11-14T16:03:02.756529Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 52.0/1
2025-11-14T16:03:02.756538Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 52 partNo 0
2025-11-14T16:03:02.756546Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 52 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 5506084 count 21 nextOffset 53 batches 11
2025-11-14T16:03:02.756547Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 52:0 appended
2025-11-14T16:03:02.756549Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[21/32] d0000000000_00000000000000000053_00000_0000000001_00000?
2025-11-14T16:03:02.756550Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.756552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000053_00000_0000000001_00000?, parameters.CurOffset=53
2025-11-14T16:03:02.756597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 53.0/1
2025-11-14T16:03:02.756603Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 53 partNo 0
2025-11-14T16:03:02.757137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 53 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 5768257 count 22 nextOffset 54 batches 11
2025-11-14T16:03:02.757147Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 53:0 appended
2025-11-14T16:03:02.757151Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[22/32] d0000000000_00000000000000000054_00000_0000000001_00000?
2025-11-14T16:03:02.757152Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.757155Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000054_00000_0000000001_00000?, parameters.CurOffset=54
2025-11-14T16:03:02.757222Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 54.0/1
2025-11-14T16:03:02.757231Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 54 partNo 0
2025-11-14T16:03:02.757241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 54 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 6030471 count 23 nextOffset 55 batches 12
2025-11-14T16:03:02.757243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 54:0 appended
2025-11-14T16:03:02.757245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[23/32] d0000000000_00000000000000000055_00000_0000000001_00000?
2025-11-14T16:03:02.757246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.757247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000055_00000_0000000001_00000?, parameters.CurOffset=55
2025-11-14T16:03:02.757299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 55.0/1
2025-11-14T16:03:02.757308Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 55 partNo 0
2025-11-14T16:03:02.757875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 55 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 6292644 count 24 nextOffset 56 batches 12
2025-11-14T16:03:02.757883Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 55:0 appended
2025-11-14T16:03:02.757887Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[24/32] d0000000000_00000000000000000056_00000_0000000001_00000?
2025-11-14T16:03:02.757890Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.757893Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000056_00000_0000000001_00000?, parameters.CurOffset=56
2025-11-14T16:03:02.757952Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 56.0/1
2025-11-14T16:03:02.757961Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 56 partNo 0
2025-11-14T16:03:02.757971Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 56 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 6554858 count 25 nextOffset 57 batches 13
2025-11-14T16:03:02.757974Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 56:0 appended
2025-11-14T16:03:02.757978Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[25/32] d0000000000_00000000000000000057_00000_0000000001_00000?
2025-11-14T16:03:02.757979Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.757982Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000057_00000_0000000001_00000?, parameters.CurOffset=57
2025-11-14T16:03:02.758035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 57.0/1
2025-11-14T16:03:02.758044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 57 partNo 0
2025-11-14T16:03:02.758772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 57 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 6817031 count 26 nextOffset 58 batches 13
2025-11-14T16:03:02.758780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 57:0 appended
2025-11-14T16:03:02.758784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[26/32] d0000000000_00000000000000000058_00000_0000000001_00000?
2025-11-14T16:03:02.758785Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.758787Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000058_00000_0000000001_00000?, parameters.CurOffset=58
2025-11-14T16:03:02.758848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 58.0/1
2025-11-14T16:03:02.758857Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 58 partNo 0
2025-11-14T16:03:02.758868Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 58 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 7079245 count 27 nextOffset 59 batches 14
2025-11-14T16:03:02.758870Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 58:0 appended
2025-11-14T16:03:02.758874Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[27/32] d0000000000_00000000000000000059_00000_0000000001_00000?
2025-11-14T16:03:02.758875Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.758878Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000059_00000_0000000001_00000?, parameters.CurOffset=59
2025-11-14T16:03:02.758930Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 59.0/1
2025-11-14T16:03:02.758939Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 59 partNo 0
2025-11-14T16:03:02.759669Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 59 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 7341418 count 28 nextOffset 60 batches 14
2025-11-14T16:03:02.759682Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 59:0 appended
2025-11-14T16:03:02.759687Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[28/32] d0000000000_00000000000000000060_00000_0000000001_00000?
2025-11-14T16:03:02.759691Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.759694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000060_00000_0000000001_00000?, parameters.CurOffset=60
2025-11-14T16:03:02.759752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 60.0/1
2025-11-14T16:03:02.759763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 60 partNo 0
2025-11-14T16:03:02.759777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 60 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 7603632 count 29 nextOffset 61 batches 15
2025-11-14T16:03:02.759778Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 60:0 appended
2025-11-14T16:03:02.759783Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[29/32] d0000000000_00000000000000000061_00000_0000000001_00000?
2025-11-14T16:03:02.759784Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.759786Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000061_00000_0000000001_00000?, parameters.CurOffset=61
2025-11-14T16:03:02.759842Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 61.0/1
2025-11-14T16:03:02.759851Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 61 partNo 0
2025-11-14T16:03:02.760645Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 61 partNo 0 FormedBlobsCount 0 NewHead: Offset 32 PartNo 0 PackedSize 7865805 count 30 nextOffset 62 batches 15
2025-11-14T16:03:02.760652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 61:0 appended
2025-11-14T16:03:02.760657Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[30/32] d0000000000_00000000000000000062_00000_0000000001_00000?
2025-11-14T16:03:02.760659Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.760662Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000062_00000_0000000001_00000?, parameters.CurOffset=62
2025-11-14T16:03:02.760723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 62.0/1
2025-11-14T16:03:02.760735Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 62 partNo 0
2025-11-14T16:03:02.771046Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 62 partNo 0 result is x0000000000_00000000000000000031_00000_0000000031_00000 size 8128009
2025-11-14T16:03:02.771073Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1104: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000031_00000_0000000031_00000 new key d0000000000_00000000000000000031_00000_0000000031_00000 size 8128009 WTime 1763136182770
2025-11-14T16:03:02.771109Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 62 partNo 0 FormedBlobsCount 1 NewHead: Offset 62 PartNo 0 PackedSize 262214 count 1 nextOffset 63 batches 1
2025-11-14T16:03:02.771115Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 62:0 appended
2025-11-14T16:03:02.771127Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[31/32] d0000000000_00000000000000000063_00000_0000000001_00000?
2025-11-14T16:03:02.771141Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:02.771146Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000063_00000_0000000001_00000?, parameters.CurOffset=63
2025-11-14T16:03:02.771238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 63.0/1
2025-11-14T16:03:02.771253Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 63 partNo 0
2025-11-14T16:03:02.771853Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 63 partNo 0 FormedBlobsCount 0 NewHead: Offset 62 PartNo 0 PackedSize 524387 count 2 nextOffset 64 batches 1
2025-11-14T16:03:02.771864Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 63:0 appended
2025-11-14T16:03:02.771906Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 62,2 HeadOffset 31 endOffset 32 curOffset 64 d0000000000_00000000000000000062_00000_0000000002_00000| size 524387 WTime 1763136182771
2025-11-14T16:03:02.772305Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951570:3565], now have 1 active actors on pipe
2025-11-14T16:03:02.772334Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.772344Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.772359Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.772379Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 31 partNo 0 count 31 size 8128009
2025-11-14T16:03:02.772387Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 62 partNo 0 count 2 size 524387
2025-11-14T16:03:02.772427Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|e11a47aa-be2143ea-d730cfc8-556313e_64 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.772439Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.772442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.772448Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.772451Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.772454Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.772464Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.772466Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.772468Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.772480Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.772520Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.772587Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.772938Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951570:3565] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|e11a47aa-be2143ea-d730cfc8-556313e_64"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|e11a47aa-be2143ea-d730cfc8-556313e_64"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 64
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.775810Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.776183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951576:3568], now have 1 active actors on pipe
2025-11-14T16:03:02.776525Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.776535Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.776551Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 64 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.776563Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.776574Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.776579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.776586Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.776588Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.776591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.776610Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 64 partNo 0
2025-11-14T16:03:02.776640Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 64 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 262214 count 1 nextOffset 65 batches 1
2025-11-14T16:03:02.776648Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.776652Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.776655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.777020Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 64,1 HeadOffset 64 endOffset 64 curOffset 65 d0000000000_00000000000000000064_00000_0000000001_00000? size 262204 WTime 1763136182776
2025-11-14T16:03:02.777058Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.777066Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 64 partNo 0 count 1 size 262204
2025-11-14T16:03:02.785292Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 31 count 31 size 8128009 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.785314Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 62 count 2 size 524387 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.785329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.785335Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 31 parts 0 suffix '0' size 8128009
2025-11-14T16:03:02.785341Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 2 parts 0 suffix '124' size 524387
2025-11-14T16:03:02.785776Z node 2 :PERSQUEUE DEBUG: partition.cpp:2289: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-14T16:03:02.785788Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.786223Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 64 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.786250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.786269Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 64 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.786273Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.786283Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.786294Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 64, partNo: 0, Offset: 64 is stored on disk
2025-11-14T16:03:02.786329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.786339Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.786342Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.786344Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.786349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.786399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.786419Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.786447Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.786461Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000032_00000_0000000001_00000?(+) to d0000000000_00000000000000000032_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786463Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000033_00000_0000000001_00000?(+) to d0000000000_00000000000000000033_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786466Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000034_00000_0000000001_00000?(+) to d0000000000_00000000000000000034_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786468Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000035_00000_0000000001_00000?(+) to d0000000000_00000000000000000035_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786471Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000036_00000_0000000001_00000?(+) to d0000000000_00000000000000000036_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786474Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000037_00000_0000000001_00000?(+) to d0000000000_00000000000000000037_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786477Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000038_00000_0000000001_00000?(+) to d0000000000_00000000000000000038_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786479Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000039_00000_0000000001_00000?(+) to d0000000000_00000000000000000039_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786484Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000040_00000_0000000001_00000?(+) to d0000000000_00000000000000000040_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786487Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000041_00000_0000000001_00000?(+) to d0000000000_00000000000000000041_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786489Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000042_00000_0000000001_00000?(+) to d0000000000_00000000000000000042_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786492Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000043_00000_0000000001_00000?(+) to d0000000000_00000000000000000043_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786491Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.786494Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000044_00000_0000000001_00000?(+) to d0000000000_00000000000000000044_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786496Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000045_00000_0000000001_00000?(+) to d0000000000_00000000000000000045_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786500Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000046_00000_0000000001_00000?(+) to d0000000000_00000000000000000046_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786502Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000047_00000_0000000001_00000?(+) to d0000000000_00000000000000000047_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786505Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000048_00000_0000000001_00000?(+) to d0000000000_00000000000000000048_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786508Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000049_00000_0000000001_00000?(+) to d0000000000_00000000000000000049_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786510Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000050_00000_0000000001_00000?(+) to d0000000000_00000000000000000050_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786512Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000051_00000_0000000001_00000?(+) to d0000000000_00000000000000000051_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786515Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000052_00000_0000000001_00000?(+) to d0000000000_00000000000000000052_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786517Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000053_00000_0000000001_00000?(+) to d0000000000_00000000000000000053_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786519Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000054_00000_0000000001_00000?(+) to d0000000000_00000000000000000054_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786521Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000055_00000_0000000001_00000?(+) to d0000000000_00000000000000000055_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786523Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000056_00000_0000000001_00000?(+) to d0000000000_00000000000000000056_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786525Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000057_00000_0000000001_00000?(+) to d0000000000_00000000000000000057_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786527Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000058_00000_0000000001_00000?(+) to d0000000000_00000000000000000058_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786529Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000059_00000_0000000001_00000?(+) to d0000000000_00000000000000000059_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786531Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000060_00000_0000000001_00000?(+) to d0000000000_00000000000000000060_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786533Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000061_00000_0000000001_00000?(+) to d0000000000_00000000000000000061_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786535Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000062_00000_0000000001_00000?(+) to d0000000000_00000000000000000062_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786537Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000063_00000_0000000001_00000?(+) to d0000000000_00000000000000000063_00000_0000000001_00000?(+)
2025-11-14T16:03:02.786539Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000031_00000_0000000001_00000|(+) to d0000000000_00000000000000000031_00000_0000000001_00000|(+)
CallPersQueueGRPC response:
2025-11-14T16:03:02.786914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951576:3568] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 64
    Offset: 64
    WriteTimestampMS: 1763136182776
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 9
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.787408Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 32 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787419Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 33 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787424Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 34 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787427Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 35 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787432Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 36 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787436Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 37 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787440Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 38 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787444Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 39 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787450Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 40 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787455Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 41 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787459Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 42 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787463Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 43 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787467Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 44 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787471Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 45 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787475Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 46 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787477Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 47 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787482Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 48 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787484Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 49 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787486Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 50 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787488Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 51 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787490Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 52 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787493Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 53 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787495Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 54 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787497Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 55 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787501Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 56 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787503Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 57 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787505Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 58 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787508Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 59 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787510Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 60 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787512Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 61 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787514Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 62 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 63 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787521Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 31 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.787535Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.787544Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.787555Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.787557Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.787559Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.787559Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 32 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787561Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.787563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.787568Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.787565Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 33 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787580Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 34 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787585Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 35 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787588Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 36 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787592Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 37 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787596Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 38 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787600Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 39 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787604Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 40 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787608Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 41 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787611Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 42 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787615Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 43 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787618Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 44 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787620Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 45 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787624Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 46 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787627Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 47 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787629Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 48 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787633Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 49 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787635Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 50 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787638Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 51 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787628Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.787640Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 52 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787645Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 53 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787648Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 54 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787651Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 55 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787654Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 56 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787656Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 57 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787659Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 58 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787662Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 59 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787665Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 60 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787669Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 61 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787672Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787675Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 63 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.787678Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 1 parts 0 suffix '124' size 262204
2025-11-14T16:03:02.788038Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951582:3572], now have 1 active actors on pipe
2025-11-14T16:03:02.788066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.788071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.788098Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|a9f0b9c-afe0269c-7da78537-76ad373e_65 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.788109Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.788111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.788115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.788117Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.788121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.788129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.788131Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.788133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.788143Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.788155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.788188Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.788406Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951582:3572] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|a9f0b9c-afe0269c-7da78537-76ad373e_65"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|a9f0b9c-afe0269c-7da78537-76ad373e_65"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 65
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.791124Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.791523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951586:3574], now have 1 active actors on pipe
2025-11-14T16:03:02.791964Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.791979Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.792000Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 65 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.792019Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.792036Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.792040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.792049Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.792052Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.792056Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.792083Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 65 partNo 0
2025-11-14T16:03:02.792119Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 65 partNo 0 FormedBlobsCount 0 NewHead: Offset 65 PartNo 0 PackedSize 262214 count 1 nextOffset 66 batches 1
2025-11-14T16:03:02.792130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.792133Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.792136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.792540Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 65,1 HeadOffset 65 endOffset 65 curOffset 66 d0000000000_00000000000000000065_00000_0000000001_00000? size 262204 WTime 1763136182792
2025-11-14T16:03:02.792593Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.792610Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 65 partNo 0 count 1 size 262204
2025-11-14T16:03:02.794951Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 65 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.794980Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.794992Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 65 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.795001Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.795013Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.795030Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 65, partNo: 0, Offset: 65 is stored on disk
2025-11-14T16:03:02.795061Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.795086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.795089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.795091Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.795094Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.795101Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.795112Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.795180Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.795549Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951586:3574] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 65
    Offset: 65
    WriteTimestampMS: 1763136182791
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.796280Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.796716Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951590:3576], now have 1 active actors on pipe
2025-11-14T16:03:02.796812Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.796823Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.796863Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9cb3d09a-d5a3ca9e-3ee675b6-fe734f10_66 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.796871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.796877Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.796883Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.796885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.796889Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.796916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.796919Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.797018Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.796923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.796937Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.796951Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.797293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951590:3576] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9cb3d09a-d5a3ca9e-3ee675b6-fe734f10_66"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9cb3d09a-d5a3ca9e-3ee675b6-fe734f10_66"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 66
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.799761Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.800331Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951593:3577], now have 1 active actors on pipe
2025-11-14T16:03:02.800706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.800721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.800746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 66 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.800761Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.800775Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.800784Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.800796Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.800799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.800804Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.800828Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 66 partNo 0
2025-11-14T16:03:02.800862Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 66 partNo 0 FormedBlobsCount 0 NewHead: Offset 66 PartNo 0 PackedSize 262214 count 1 nextOffset 67 batches 1
2025-11-14T16:03:02.800868Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.800871Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.800874Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.801015Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 66,1 HeadOffset 66 endOffset 66 curOffset 67 d0000000000_00000000000000000066_00000_0000000001_00000? size 262204 WTime 1763136182800
2025-11-14T16:03:02.801050Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.801061Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 66 partNo 0 count 1 size 262204
2025-11-14T16:03:02.809562Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 66 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.809601Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.809616Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 66 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.809628Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.809641Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.809653Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 66, partNo: 0, Offset: 66 is stored on disk
2025-11-14T16:03:02.809697Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.809707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.809710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.809714Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.809720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.809731Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.809746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.809855Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.810263Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951593:3577] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 66
    Offset: 66
    WriteTimestampMS: 1763136182800
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 8
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.811105Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.811627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951598:3580], now have 1 active actors on pipe
2025-11-14T16:03:02.811689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.811703Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.811743Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b84c500b-2ab5ca4-5525c6d0-d6ea83de_67 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.811763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.811768Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.811774Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.811777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.811782Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.811810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.811820Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.811824Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.811840Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.811863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.811927Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.812191Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951598:3580] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b84c500b-2ab5ca4-5525c6d0-d6ea83de_67"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b84c500b-2ab5ca4-5525c6d0-d6ea83de_67"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 67
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.817035Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.817622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951601:3581], now have 1 active actors on pipe
2025-11-14T16:03:02.818050Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.818071Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.818102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 67 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.818124Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.818145Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.818154Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.818166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.818174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.818180Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.818209Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 67 partNo 0
2025-11-14T16:03:02.818259Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 67 partNo 0 FormedBlobsCount 0 NewHead: Offset 67 PartNo 0 PackedSize 262214 count 1 nextOffset 68 batches 1
2025-11-14T16:03:02.818274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.818278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.818281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.818415Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 67,1 HeadOffset 67 endOffset 67 curOffset 68 d0000000000_00000000000000000067_00000_0000000001_00000? size 262204 WTime 1763136182817
2025-11-14T16:03:02.818738Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.818762Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 67 partNo 0 count 1 size 262204
2025-11-14T16:03:02.819026Z node 2 :PERSQUEUE DEBUG: partition.cpp:2289: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-14T16:03:02.820829Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 67 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.820863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.820872Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 67 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.820883Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.820892Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.820925Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 67, partNo: 0, Offset: 67 is stored on disk
2025-11-14T16:03:02.820956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.820960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.820962Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.820964Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.820966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.820975Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.820986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.821091Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.821447Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951601:3581] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 67
    Offset: 67
    WriteTimestampMS: 1763136182817
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.823921Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.824418Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951605:3583], now have 1 active actors on pipe
2025-11-14T16:03:02.824517Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.824533Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.824572Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|533ec379-bc989447-eb2a9c7f-dbb3559b_68 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.824587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.824591Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.824598Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.824607Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.824611Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.824622Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.824625Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.824627Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.824646Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.824662Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.824760Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.825108Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951605:3583] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|533ec379-bc989447-eb2a9c7f-dbb3559b_68"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|533ec379-bc989447-eb2a9c7f-dbb3559b_68"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 68
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.828136Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.830895Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951608:3584], now have 1 active actors on pipe
2025-11-14T16:03:02.831425Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.831445Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.831476Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 68 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.831498Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.831523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.831531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.831543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.831549Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.831556Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.831604Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 68 partNo 0
2025-11-14T16:03:02.831655Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 68 partNo 0 FormedBlobsCount 0 NewHead: Offset 68 PartNo 0 PackedSize 262214 count 1 nextOffset 69 batches 1
2025-11-14T16:03:02.831668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.831671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.831674Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.831803Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 68,1 HeadOffset 68 endOffset 68 curOffset 69 d0000000000_00000000000000000068_00000_0000000001_00000? size 262204 WTime 1763136182830
2025-11-14T16:03:02.831868Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.831881Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 68 partNo 0 count 1 size 262204
2025-11-14T16:03:02.834687Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 68 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.834726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.834740Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 68 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.835680Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:490: [72075186224037892][Partition][0][StateIdle] Received TPartition::Handle TEvHandleWriteResponse.
2025-11-14T16:03:02.835716Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.835733Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.835753Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 68, partNo: 0, Offset: 68 is stored on disk
2025-11-14T16:03:02.835805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.835818Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.835825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.835829Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.835832Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.835844Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.835866Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.835982Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.836395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951608:3584] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 68
    Offset: 68
    WriteTimestampMS: 1763136182830
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 4
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.837262Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.837945Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951612:3586], now have 1 active actors on pipe
2025-11-14T16:03:02.838078Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.838093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.838130Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9ef4e40a-3a10367e-4a092c9d-d06dd6e9_69 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.838146Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.838157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.838164Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.838166Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.838170Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.838181Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.838189Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.838195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.838232Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.838253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.838353Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.838646Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951612:3586] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9ef4e40a-3a10367e-4a092c9d-d06dd6e9_69"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9ef4e40a-3a10367e-4a092c9d-d06dd6e9_69"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 69
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.841052Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.841636Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951615:3587], now have 1 active actors on pipe
2025-11-14T16:03:02.842115Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.842135Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.842161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 69 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.842174Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.842194Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.842203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.842215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.842228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.842235Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.842269Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 69 partNo 0
2025-11-14T16:03:02.842315Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 69 partNo 0 FormedBlobsCount 0 NewHead: Offset 69 PartNo 0 PackedSize 262214 count 1 nextOffset 70 batches 1
2025-11-14T16:03:02.842329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.842332Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.842337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.842468Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 69,1 HeadOffset 69 endOffset 69 curOffset 70 d0000000000_00000000000000000069_00000_0000000001_00000? size 262204 WTime 1763136182842
2025-11-14T16:03:02.842531Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.842548Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 69 partNo 0 count 1 size 262204
2025-11-14T16:03:02.845386Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 69 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.845415Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.845431Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 69 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.845440Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.845450Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.845468Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 69, partNo: 0, Offset: 69 is stored on disk
2025-11-14T16:03:02.845516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.845524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.845526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.845529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.845531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.845541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.845552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.845648Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.846010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951615:3587] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 69
    Offset: 69
    WriteTimestampMS: 1763136182842
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.847153Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.847589Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951619:3589], now have 1 active actors on pipe
2025-11-14T16:03:02.847796Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.847810Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.847864Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ee16f04e-e887ec2b-c6763d02-8b233834_70 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.847879Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.847886Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.847892Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.847894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.847899Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.847910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.847917Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.847920Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.847937Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.847954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.848024Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.848289Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951619:3589] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ee16f04e-e887ec2b-c6763d02-8b233834_70"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ee16f04e-e887ec2b-c6763d02-8b233834_70"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 70
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.850586Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.851067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951623:3591], now have 1 active actors on pipe
2025-11-14T16:03:02.851467Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.851491Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.851527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 70 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.851549Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.851569Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.851577Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.851587Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.851589Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.851595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.851625Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 70 partNo 0
2025-11-14T16:03:02.851660Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 70 partNo 0 FormedBlobsCount 0 NewHead: Offset 70 PartNo 0 PackedSize 262214 count 1 nextOffset 71 batches 1
2025-11-14T16:03:02.851667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.851671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.851673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.851804Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 70,1 HeadOffset 70 endOffset 70 curOffset 71 d0000000000_00000000000000000070_00000_0000000001_00000? size 262204 WTime 1763136182851
2025-11-14T16:03:02.851844Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.851855Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 70 partNo 0 count 1 size 262204
2025-11-14T16:03:02.854700Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 70 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.854734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.854753Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 70 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.854757Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.854767Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.854781Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 70, partNo: 0, Offset: 70 is stored on disk
2025-11-14T16:03:02.854826Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.854835Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.854837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.854839Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.854842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.854850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.854864Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.854971Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.855378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951623:3591] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 70
    Offset: 70
    WriteTimestampMS: 1763136182851
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.856084Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.856602Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951627:3593], now have 1 active actors on pipe
2025-11-14T16:03:02.856685Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.856698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.856739Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|7b0b6d1b-80d97fea-771be5-5cb42048_71 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.856755Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.856759Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.856765Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.856767Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.856771Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.856783Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.856791Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.856794Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.856809Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.856824Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.856886Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|7b0b6d1b-80d97fea-771be5-5cb42048_71"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
2025-11-14T16:03:02.857345Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951627:3593] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|7b0b6d1b-80d97fea-771be5-5cb42048_71"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 71
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.859820Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.860223Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951630:3594], now have 1 active actors on pipe
2025-11-14T16:03:02.860701Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.860719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.860743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 71 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.860760Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.860780Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.860789Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.860800Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.860803Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.860808Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.860839Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 71 partNo 0
2025-11-14T16:03:02.860877Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 71 partNo 0 FormedBlobsCount 0 NewHead: Offset 71 PartNo 0 PackedSize 262214 count 1 nextOffset 72 batches 1
2025-11-14T16:03:02.860885Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.860887Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.860890Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.861034Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 71,1 HeadOffset 71 endOffset 71 curOffset 72 d0000000000_00000000000000000071_00000_0000000001_00000? size 262204 WTime 1763136182860
2025-11-14T16:03:02.861071Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.861083Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 71 partNo 0 count 1 size 262204
2025-11-14T16:03:02.863801Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 71 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.863848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.863874Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.863890Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.863905Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 71, partNo: 0, Offset: 71 is stored on disk
2025-11-14T16:03:02.863945Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.863953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.863955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.863958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.863960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.863969Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.863981Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.864035Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 71 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.864144Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.864746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951630:3594] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 71
    Offset: 71
    WriteTimestampMS: 1763136182860
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.867493Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.867988Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951634:3596], now have 1 active actors on pipe
2025-11-14T16:03:02.868141Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.868154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.868192Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f549eb57-83a35b7a-b2b7c8b0-6140c991_72 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.868209Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.868214Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.868219Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.868223Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.868228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.868240Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.868248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.868251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.868270Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.868294Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.868399Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.868806Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951634:3596] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f549eb57-83a35b7a-b2b7c8b0-6140c991_72"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f549eb57-83a35b7a-b2b7c8b0-6140c991_72"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 72
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.871339Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.871852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951637:3597], now have 1 active actors on pipe
2025-11-14T16:03:02.872264Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.872283Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.872308Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 72 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.872321Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.872337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.872340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.872353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.872355Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.872362Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.872392Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 72 partNo 0
2025-11-14T16:03:02.872432Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 72 partNo 0 FormedBlobsCount 0 NewHead: Offset 72 PartNo 0 PackedSize 262214 count 1 nextOffset 73 batches 1
2025-11-14T16:03:02.872440Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.872442Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.872444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.872561Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 72,1 HeadOffset 72 endOffset 72 curOffset 73 d0000000000_00000000000000000072_00000_0000000001_00000? size 262204 WTime 1763136182872
2025-11-14T16:03:02.872603Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.872615Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 72 partNo 0 count 1 size 262204
2025-11-14T16:03:02.875007Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 72 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.875037Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.875051Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 72 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.875062Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.875072Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.875085Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 72, partNo: 0, Offset: 72 is stored on disk
2025-11-14T16:03:02.875124Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.875128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.875129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.875132Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.875134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.875144Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.875155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.875232Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.875582Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951637:3597] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 72
    Offset: 72
    WriteTimestampMS: 1763136182872
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.876424Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.876842Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951641:3599], now have 1 active actors on pipe
2025-11-14T16:03:02.877048Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.877065Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.877099Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|586fb894-9748c77f-fe3a6fca-52263332_73 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.877115Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.877125Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.877130Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.877136Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.877141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.877151Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.877157Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.877160Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.877187Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.877207Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.877312Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|586fb894-9748c77f-fe3a6fca-52263332_73"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|586fb894-9748c77f-fe3a6fca-52263332_73"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 73
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.878745Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951641:3599] destroyed
2025-11-14T16:03:02.879996Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.880514Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951644:3600], now have 1 active actors on pipe
2025-11-14T16:03:02.880870Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.880893Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.880935Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 73 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.880963Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.880978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.880988Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.880999Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.881006Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.881013Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.881051Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 73 partNo 0
2025-11-14T16:03:02.881101Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 73 partNo 0 FormedBlobsCount 0 NewHead: Offset 73 PartNo 0 PackedSize 262214 count 1 nextOffset 74 batches 1
2025-11-14T16:03:02.881113Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.881116Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.881119Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.881274Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 73,1 HeadOffset 73 endOffset 73 curOffset 74 d0000000000_00000000000000000073_00000_0000000001_00000? size 262204 WTime 1763136182880
2025-11-14T16:03:02.881326Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.881341Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 73 partNo 0 count 1 size 262204
2025-11-14T16:03:02.883822Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 73 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.883845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.883870Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.883880Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.883890Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 73, partNo: 0, Offset: 73 is stored on disk
2025-11-14T16:03:02.883923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.883926Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.883928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.883930Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.883933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.883941Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.883954Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.884021Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 73 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.884092Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 73
    Offset: 73
    WriteTimestampMS: 1763136182880
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.884849Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951644:3600] destroyed
2025-11-14T16:03:02.885289Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.885779Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951648:3602], now have 1 active actors on pipe
2025-11-14T16:03:02.885900Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.885913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.885944Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|d5d57d51-a7e89779-c8cf8559-2d513ccc_74 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.885958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.885961Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.885966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.885967Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.885971Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.885981Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.885986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.885989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.886003Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.886016Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.886096Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|d5d57d51-a7e89779-c8cf8559-2d513ccc_74"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|d5d57d51-a7e89779-c8cf8559-2d513ccc_74"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 74
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.887764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951648:3602] destroyed
2025-11-14T16:03:02.888587Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.889052Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951651:3603], now have 1 active actors on pipe
2025-11-14T16:03:02.889463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.889478Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.889502Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 74 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.889513Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.889526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.889530Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.889539Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.889541Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.889550Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.889576Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 74 partNo 0
2025-11-14T16:03:02.889612Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 74 partNo 0 FormedBlobsCount 0 NewHead: Offset 74 PartNo 0 PackedSize 262214 count 1 nextOffset 75 batches 1
2025-11-14T16:03:02.889632Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.889635Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.889637Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.889753Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 74,1 HeadOffset 74 endOffset 74 curOffset 75 d0000000000_00000000000000000074_00000_0000000001_00000? size 262204 WTime 1763136182889
2025-11-14T16:03:02.889791Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.889803Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 74 partNo 0 count 1 size 262204
2025-11-14T16:03:02.892011Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 74 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.892039Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.892058Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 74 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.892061Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.892073Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.892084Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 74, partNo: 0, Offset: 74 is stored on disk
2025-11-14T16:03:02.892118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.892121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.892123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.892126Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.892128Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.892136Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.892147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.892382Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 74
    Offset: 74
    WriteTimestampMS: 1763136182889
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.893102Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951651:3603] destroyed
2025-11-14T16:03:02.893430Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.893793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951655:3605], now have 1 active actors on pipe
2025-11-14T16:03:02.893920Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.893932Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.893964Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|ef726504-f06b0b13-9143a462-eabb0258_75 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.893974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.893978Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.893983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.893986Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.893991Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.894003Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.894005Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.894008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.894021Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.894033Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.894075Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|ef726504-f06b0b13-9143a462-eabb0258_75"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
2025-11-14T16:03:02.894885Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951655:3605] destroyed
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|ef726504-f06b0b13-9143a462-eabb0258_75"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 75
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.896532Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.897114Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951658:3606], now have 1 active actors on pipe
2025-11-14T16:03:02.897642Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.897664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.897687Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 75 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.897703Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.897723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.897731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.897745Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.897751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.897757Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.897785Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 75 partNo 0
2025-11-14T16:03:02.897919Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 75 partNo 0 FormedBlobsCount 0 NewHead: Offset 75 PartNo 0 PackedSize 262214 count 1 nextOffset 76 batches 1
2025-11-14T16:03:02.897934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.897936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.897939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.898078Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 75,1 HeadOffset 75 endOffset 75 curOffset 76 d0000000000_00000000000000000075_00000_0000000001_00000? size 262204 WTime 1763136182897
2025-11-14T16:03:02.898123Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.898143Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 75 partNo 0 count 1 size 262204
2025-11-14T16:03:02.901074Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 75 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.901114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.901145Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.901148Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 75 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.901159Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.901202Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 75, partNo: 0, Offset: 75 is stored on disk
2025-11-14T16:03:02.901264Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.901269Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.901271Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.901275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.901277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.901291Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.901309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.901400Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 75
    Offset: 75
    WriteTimestampMS: 1763136182897
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-14T16:03:02.901852Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951658:3606] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.904347Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.904851Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951662:3608], now have 1 active actors on pipe
2025-11-14T16:03:02.904990Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.905004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.905054Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|eb3f4d1f-89e9356c-b9f45e7c-80483c1f_76 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.905068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.905072Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.905080Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.905082Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.905087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.905102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.905105Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.905108Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.905131Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.905155Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.905231Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.905498Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951662:3608] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|eb3f4d1f-89e9356c-b9f45e7c-80483c1f_76"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|eb3f4d1f-89e9356c-b9f45e7c-80483c1f_76"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 76
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.908637Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.909203Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951665:3609], now have 1 active actors on pipe
2025-11-14T16:03:02.909594Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.909618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.909639Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 76 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.909650Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.909662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.909665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.909673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.909680Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.909684Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.909708Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 76 partNo 0
2025-11-14T16:03:02.909744Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 76 partNo 0 FormedBlobsCount 0 NewHead: Offset 76 PartNo 0 PackedSize 262214 count 1 nextOffset 77 batches 1
2025-11-14T16:03:02.909758Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.909761Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.909763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.909867Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 76,1 HeadOffset 76 endOffset 76 curOffset 77 d0000000000_00000000000000000076_00000_0000000001_00000? size 262204 WTime 1763136182909
2025-11-14T16:03:02.909900Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.909908Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 76 partNo 0 count 1 size 262204
2025-11-14T16:03:02.912404Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 76 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.912437Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.912458Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.912459Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 76 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.912469Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.912481Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 76, partNo: 0, Offset: 76 is stored on disk
2025-11-14T16:03:02.912511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.912519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.912520Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.912522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.912524Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.912531Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.912540Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.912617Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.912919Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951665:3609] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 76
    Offset: 76
    WriteTimestampMS: 1763136182909
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.913613Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.914040Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951669:3611], now have 1 active actors on pipe
2025-11-14T16:03:02.914265Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.914293Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.914333Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b9c1fb46-c2278b08-58dab5a6-fa9d7239_77 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.914349Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.914353Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.914364Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.914367Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.914373Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.914383Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.914394Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.914397Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.914411Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.914429Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.914516Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.914739Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951669:3611] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b9c1fb46-c2278b08-58dab5a6-fa9d7239_77"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b9c1fb46-c2278b08-58dab5a6-fa9d7239_77"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 77
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.916723Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.917157Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951672:3612], now have 1 active actors on pipe
2025-11-14T16:03:02.917488Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.917503Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.917523Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 77 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.917538Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.917554Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.917563Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.917572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.917575Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.917580Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.917607Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 77 partNo 0
2025-11-14T16:03:02.917648Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 77 partNo 0 FormedBlobsCount 0 NewHead: Offset 77 PartNo 0 PackedSize 262214 count 1 nextOffset 78 batches 1
2025-11-14T16:03:02.917660Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.917663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.917667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.917792Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 77,1 HeadOffset 77 endOffset 77 curOffset 78 d0000000000_00000000000000000077_00000_0000000001_00000? size 262204 WTime 1763136182917
2025-11-14T16:03:02.917836Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.917855Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 77 partNo 0 count 1 size 262204
2025-11-14T16:03:02.919380Z node 2 :PERSQUEUE DEBUG: partition.cpp:2289: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-14T16:03:02.926808Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 77 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.926852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.926865Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 77 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.926876Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.926889Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.926905Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 77, partNo: 0, Offset: 77 is stored on disk
2025-11-14T16:03:02.926957Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.926966Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.926969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.926972Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.926974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.926986Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.927004Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.927133Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.927624Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951672:3612] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 77
    Offset: 77
    WriteTimestampMS: 1763136182917
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 9
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.928469Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.928969Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951676:3614], now have 1 active actors on pipe
2025-11-14T16:03:02.929147Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.929186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.929248Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|d9531cd8-a492768d-88b0323a-180fed58_78 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.929270Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.929281Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.929289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.929299Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.929303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.929318Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.929327Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.929331Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.929352Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.929378Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.929486Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.929837Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951676:3614] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|d9531cd8-a492768d-88b0323a-180fed58_78"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|d9531cd8-a492768d-88b0323a-180fed58_78"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 78
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.932210Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.932809Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951679:3615], now have 1 active actors on pipe
2025-11-14T16:03:02.933309Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.933403Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.933435Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 78 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.933461Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.933480Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.933485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.933499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.933501Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.933506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.933545Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 78 partNo 0
2025-11-14T16:03:02.933596Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 78 partNo 0 FormedBlobsCount 0 NewHead: Offset 78 PartNo 0 PackedSize 262214 count 1 nextOffset 79 batches 1
2025-11-14T16:03:02.933605Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.933614Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.933618Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.933778Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 78,1 HeadOffset 78 endOffset 78 curOffset 79 d0000000000_00000000000000000078_00000_0000000001_00000? size 262204 WTime 1763136182933
2025-11-14T16:03:02.933834Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.933855Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 78 partNo 0 count 1 size 262204
2025-11-14T16:03:02.936822Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 78 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.936863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.936882Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 78 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.936890Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.936929Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.936949Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 78, partNo: 0, Offset: 78 is stored on disk
2025-11-14T16:03:02.936997Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.937008Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.937011Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.937014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.937016Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.937044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.937066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.937211Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.937663Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951679:3615] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 78
    Offset: 78
    WriteTimestampMS: 1763136182933
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.938675Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.939225Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951683:3617], now have 1 active actors on pipe
2025-11-14T16:03:02.939410Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.939431Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.939468Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f2f65dca-8860d295-bb698b0e-f293df5e_79 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.939488Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.939492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.939500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.939503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.939507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.939519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.939528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.939533Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.939551Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.939573Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.939673Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f2f65dca-8860d295-bb698b0e-f293df5e_79"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f2f65dca-8860d295-bb698b0e-f293df5e_79"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 79
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.941086Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951683:3617] destroyed
2025-11-14T16:03:02.942521Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.943058Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951686:3618], now have 1 active actors on pipe
2025-11-14T16:03:02.943682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.943696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.943723Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 79 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.943734Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.943751Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.943756Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.943769Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.943772Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.943778Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.943812Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 79 partNo 0
2025-11-14T16:03:02.943849Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 79 partNo 0 FormedBlobsCount 0 NewHead: Offset 79 PartNo 0 PackedSize 262214 count 1 nextOffset 80 batches 1
2025-11-14T16:03:02.943857Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.943860Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.943864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.943989Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 79,1 HeadOffset 79 endOffset 79 curOffset 80 d0000000000_00000000000000000079_00000_0000000001_00000? size 262204 WTime 1763136182943
2025-11-14T16:03:02.944028Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.944040Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 79 partNo 0 count 1 size 262204
2025-11-14T16:03:02.946817Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 79 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.946842Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.946867Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.946878Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.946889Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 79, partNo: 0, Offset: 79 is stored on disk
2025-11-14T16:03:02.946925Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.946928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.946932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.946935Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.946937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.946948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.946960Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.947015Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 79 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.947131Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 79
    Offset: 79
    WriteTimestampMS: 1763136182943
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.947911Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951686:3618] destroyed
2025-11-14T16:03:02.948390Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.948918Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951690:3620], now have 1 active actors on pipe
2025-11-14T16:03:02.949030Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.949041Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.949082Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|1b87d341-91706c4-b668c3a2-376e523d_80 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.949093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.949096Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.949102Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.949104Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.949107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.949118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.949121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.949123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.949139Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.949153Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.949228Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|1b87d341-91706c4-b668c3a2-376e523d_80"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|1b87d341-91706c4-b668c3a2-376e523d_80"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 80
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.950618Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951690:3620] destroyed
2025-11-14T16:03:02.951828Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.952411Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951696:3624], now have 1 active actors on pipe
2025-11-14T16:03:02.952871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.952882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.952922Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 80 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.952938Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.952954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.952958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.952971Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.952974Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.952979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.953021Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 80 partNo 0
2025-11-14T16:03:02.953078Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 80 partNo 0 FormedBlobsCount 0 NewHead: Offset 80 PartNo 0 PackedSize 262214 count 1 nextOffset 81 batches 1
2025-11-14T16:03:02.953086Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.953089Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.953095Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.953233Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 80,1 HeadOffset 80 endOffset 80 curOffset 81 d0000000000_00000000000000000080_00000_0000000001_00000? size 262204 WTime 1763136182953
2025-11-14T16:03:02.953270Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.953282Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 80 partNo 0 count 1 size 262204
2025-11-14T16:03:02.955554Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 80 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.955578Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.955599Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.955599Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 80 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.955608Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.955618Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 80, partNo: 0, Offset: 80 is stored on disk
2025-11-14T16:03:02.955654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.955658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.955659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.955662Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.955664Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.955674Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.955684Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.955800Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 80
    Offset: 80
    WriteTimestampMS: 1763136182952
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-14T16:03:02.956185Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951696:3624] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.956810Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.957456Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951700:3626], now have 1 active actors on pipe
2025-11-14T16:03:02.957581Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.957593Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.957651Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|62177ba0-599c83bc-4f8afdc-f17dce4_81 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.957667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.957676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.957683Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.957691Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.957695Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.957707Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.957711Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.957720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.957752Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.957774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.958582Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|62177ba0-599c83bc-4f8afdc-f17dce4_81"
    Status: Active
    SeqNo: 0
  }
}
2025-11-14T16:03:02.958956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951700:3626] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|62177ba0-599c83bc-4f8afdc-f17dce4_81"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 81
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.961648Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.962292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951703:3627], now have 1 active actors on pipe
2025-11-14T16:03:02.962605Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.962622Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.962644Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 81 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.962662Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.962678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.962687Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.962698Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.962706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.962710Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.962741Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 81 partNo 0
2025-11-14T16:03:02.962783Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 81 partNo 0 FormedBlobsCount 0 NewHead: Offset 81 PartNo 0 PackedSize 262214 count 1 nextOffset 82 batches 1
2025-11-14T16:03:02.962799Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.962802Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.962805Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.962940Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 81,1 HeadOffset 81 endOffset 81 curOffset 82 d0000000000_00000000000000000081_00000_0000000001_00000? size 262204 WTime 1763136182962
2025-11-14T16:03:02.962983Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.962999Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 81 partNo 0 count 1 size 262204
2025-11-14T16:03:02.966282Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.966030Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 81 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.966069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.966092Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.966101Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.966111Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 81, partNo: 0, Offset: 81 is stored on disk
2025-11-14T16:03:02.966153Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.966156Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.966159Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.966162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.966167Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.966174Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 81 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.966176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.966187Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
CallPersQueueGRPC response:
2025-11-14T16:03:02.966627Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951703:3627] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 81
    Offset: 81
    WriteTimestampMS: 1763136182962
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.967390Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.967768Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951707:3629], now have 1 active actors on pipe
2025-11-14T16:03:02.967884Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.967896Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.967928Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|123563ac-a81060e8-db883a3e-897e4c8c_82 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.967942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.967947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.967952Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.967955Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.967958Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.967969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.967976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.967979Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.967993Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.968010Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.968059Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.968316Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951707:3629] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|123563ac-a81060e8-db883a3e-897e4c8c_82"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|123563ac-a81060e8-db883a3e-897e4c8c_82"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 82
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.970435Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.970878Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951710:3630], now have 1 active actors on pipe
2025-11-14T16:03:02.971317Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.971335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.971357Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 82 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.971373Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.971391Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.971395Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.971407Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.971416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.971422Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.971454Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 82 partNo 0
2025-11-14T16:03:02.971511Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 82 partNo 0 FormedBlobsCount 0 NewHead: Offset 82 PartNo 0 PackedSize 262214 count 1 nextOffset 83 batches 1
2025-11-14T16:03:02.971523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.971526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.971529Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.971659Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 82,1 HeadOffset 82 endOffset 82 curOffset 83 d0000000000_00000000000000000082_00000_0000000001_00000? size 262204 WTime 1763136182971
2025-11-14T16:03:02.971715Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.971732Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 82 partNo 0 count 1 size 262204
2025-11-14T16:03:02.974182Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 82 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.974211Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.974230Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 82 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.974232Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.974241Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.974250Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 82, partNo: 0, Offset: 82 is stored on disk
2025-11-14T16:03:02.974288Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.974296Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.974298Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.974301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.974303Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.974311Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.974322Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.974395Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.974688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951710:3630] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 82
    Offset: 82
    WriteTimestampMS: 1763136182971
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.975487Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.975793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951719:3635], now have 1 active actors on pipe
2025-11-14T16:03:02.975871Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.975882Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.975913Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9ef9f0e7-52b04e12-a9ebc04c-f9a2022c_83 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.975923Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.975929Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.975933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.975934Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.975937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.975944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.975947Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.975949Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.975958Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.975972Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.976016Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.976186Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951719:3635] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9ef9f0e7-52b04e12-a9ebc04c-f9a2022c_83"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9ef9f0e7-52b04e12-a9ebc04c-f9a2022c_83"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 83
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.978273Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.978543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951722:3636], now have 1 active actors on pipe
2025-11-14T16:03:02.978914Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.978924Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.978947Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 83 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.978958Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.978969Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.978976Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.978983Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.978985Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.978989Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.979012Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 83 partNo 0
2025-11-14T16:03:02.979043Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 83 partNo 0 FormedBlobsCount 0 NewHead: Offset 83 PartNo 0 PackedSize 262214 count 1 nextOffset 84 batches 1
2025-11-14T16:03:02.979064Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.979067Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.979069Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.979174Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 83,1 HeadOffset 83 endOffset 83 curOffset 84 d0000000000_00000000000000000083_00000_0000000001_00000? size 262204 WTime 1763136182978
2025-11-14T16:03:02.979221Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.979233Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 83 partNo 0 count 1 size 262204
2025-11-14T16:03:02.981404Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 83 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.981430Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.981448Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.981449Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 83 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.981460Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.981468Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 83, partNo: 0, Offset: 83 is stored on disk
2025-11-14T16:03:02.981497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.981506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.981508Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.981510Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.981512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.981519Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.981527Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.981621Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.981913Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951722:3636] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 83
    Offset: 83
    WriteTimestampMS: 1763136182978
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.982816Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.983173Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951735:3643], now have 1 active actors on pipe
2025-11-14T16:03:02.983284Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.983292Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.983318Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|7d7c276-8a897af-fb6295eb-a126b850_84 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.983337Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.983340Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.983343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.983345Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.983347Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.983354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.983356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.983357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.983365Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.983373Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.983447Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.983696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951735:3643] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|7d7c276-8a897af-fb6295eb-a126b850_84"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|7d7c276-8a897af-fb6295eb-a126b850_84"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 84
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.985833Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.986217Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951738:3644], now have 1 active actors on pipe
2025-11-14T16:03:02.986603Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.986611Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.986626Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 84 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.986632Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.986641Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.986647Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.986654Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.986655Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.986658Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.986677Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 84 partNo 0
2025-11-14T16:03:02.986710Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 84 partNo 0 FormedBlobsCount 0 NewHead: Offset 84 PartNo 0 PackedSize 262214 count 1 nextOffset 85 batches 1
2025-11-14T16:03:02.986719Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.986720Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.986721Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.986827Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 84,1 HeadOffset 84 endOffset 84 curOffset 85 d0000000000_00000000000000000084_00000_0000000001_00000? size 262204 WTime 1763136182986
2025-11-14T16:03:02.986874Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.986888Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 84 partNo 0 count 1 size 262204
2025-11-14T16:03:02.989538Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 84 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.989570Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.989585Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 84 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.989588Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.989600Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.989609Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 84, partNo: 0, Offset: 84 is stored on disk
2025-11-14T16:03:02.989651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.989659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.989661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.989665Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.989667Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.989675Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.989688Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.989779Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.990107Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951738:3644] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 84
    Offset: 84
    WriteTimestampMS: 1763136182986
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.990899Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.991329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951742:3646], now have 1 active actors on pipe
2025-11-14T16:03:02.991434Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.991446Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.991485Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|1ea49654-28c2ff93-7e35a645-82d63419_85 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.991497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.991500Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.991505Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.991511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.991515Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.991525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.991528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.991531Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.991544Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.991561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.991615Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:02.991807Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951742:3646] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|1ea49654-28c2ff93-7e35a645-82d63419_85"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|1ea49654-28c2ff93-7e35a645-82d63419_85"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 85
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:02.993946Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.994332Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951745:3647], now have 1 active actors on pipe
2025-11-14T16:03:02.994708Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.994721Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.994738Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 85 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:02.994748Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:02.994760Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.994763Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.994770Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.994777Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.994781Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.994812Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 85 partNo 0
2025-11-14T16:03:02.994853Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 85 partNo 0 FormedBlobsCount 0 NewHead: Offset 85 PartNo 0 PackedSize 262214 count 1 nextOffset 86 batches 1
2025-11-14T16:03:02.994861Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.994863Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.994864Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.994961Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 85,1 HeadOffset 85 endOffset 85 curOffset 86 d0000000000_00000000000000000085_00000_0000000001_00000? size 262204 WTime 1763136182994
2025-11-14T16:03:02.994995Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:02.995005Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 85 partNo 0 count 1 size 262204
2025-11-14T16:03:02.997394Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 85 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:02.997425Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:02.997441Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 85 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:02.997445Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:02.997453Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:02.997462Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 85, partNo: 0, Offset: 85 is stored on disk
2025-11-14T16:03:02.997498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.997509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.997511Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.997523Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:02.997525Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.997534Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:02.997543Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.997623Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.997943Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951745:3647] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 85
    Offset: 85
    WriteTimestampMS: 1763136182994
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:02.998634Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:02.999067Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612240639951749:3649], now have 1 active actors on pipe
2025-11-14T16:03:02.999124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:02.999136Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:02.999172Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|f0ba4db9-14ee1b2e-9090de12-10bacdc4_86 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:02.999197Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:02.999205Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:02.999210Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:02.999212Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:02.999215Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:02.999227Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:02.999234Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:02.999237Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:02.999249Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:02.999274Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:02.999365Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:02.999625Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612240639951749:3649] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|f0ba4db9-14ee1b2e-9090de12-10bacdc4_86"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|f0ba4db9-14ee1b2e-9090de12-10bacdc4_86"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 86
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:03.001891Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:03.002262Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612244934919048:3650], now have 1 active actors on pipe
2025-11-14T16:03:03.002715Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:03.002746Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:03.002766Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 86 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:03.002783Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:03.002812Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.002822Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:03.002834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:03.002836Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.002841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:03.002868Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 86 partNo 0
2025-11-14T16:03:03.002921Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 86 partNo 0 FormedBlobsCount 0 NewHead: Offset 86 PartNo 0 PackedSize 262214 count 1 nextOffset 87 batches 1
2025-11-14T16:03:03.002933Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:03.002936Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:03.002938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.003076Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 86,1 HeadOffset 86 endOffset 86 curOffset 87 d0000000000_00000000000000000086_00000_0000000001_00000? size 262204 WTime 1763136183001
2025-11-14T16:03:03.003136Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:03.003148Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 86 partNo 0 count 1 size 262204
2025-11-14T16:03:03.005677Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 86 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.005715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:03.005718Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 86 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.005734Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:03.005747Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:03.005761Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 86, partNo: 0, Offset: 86 is stored on disk
2025-11-14T16:03:03.005801Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.005808Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.005811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.005814Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.005816Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.005823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:03.005834Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:03.005916Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:03.006253Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612244934919048:3650] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 86
    Offset: 86
    WriteTimestampMS: 1763136183001
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 3
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:03.006854Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:03.007148Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612244934919052:3652], now have 1 active actors on pipe
2025-11-14T16:03:03.007220Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:03.007230Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:03.007261Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|beacdf44-3b0902e7-3bb0bd20-2b35ba7_87 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:03.007275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.007278Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:03.007282Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:03.007284Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.007287Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:03.007295Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:03.007297Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:03.007301Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.007312Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:03.007330Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:03.007373Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:03.007561Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612244934919052:3652] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|beacdf44-3b0902e7-3bb0bd20-2b35ba7_87"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|beacdf44-3b0902e7-3bb0bd20-2b35ba7_87"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 87
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:03.009776Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:03.010049Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612244934919055:3653], now have 1 active actors on pipe
2025-11-14T16:03:03.010430Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:03.010444Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:03.010463Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 87 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:03.010477Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:03.010492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.010499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:03.010506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:03.010509Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.010512Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:03.010534Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 87 partNo 0
2025-11-14T16:03:03.010568Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 87 partNo 0 FormedBlobsCount 0 NewHead: Offset 87 PartNo 0 PackedSize 262214 count 1 nextOffset 88 batches 1
2025-11-14T16:03:03.010579Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:03.010581Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:03.010584Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.010687Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 87,1 HeadOffset 87 endOffset 87 curOffset 88 d0000000000_00000000000000000087_00000_0000000001_00000? size 262204 WTime 1763136183010
2025-11-14T16:03:03.010724Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:03.010740Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 87 partNo 0 count 1 size 262204
2025-11-14T16:03:03.012816Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 87 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.012840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:03.012859Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:03.012868Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 87 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.012871Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:03.012882Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 87, partNo: 0, Offset: 87 is stored on disk
2025-11-14T16:03:03.012927Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.012937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.012938Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.012940Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.012942Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.012949Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:03.012959Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:03.013037Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 87
    Offset: 87
    WriteTimestampMS: 1763136183010
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-14T16:03:03.013377Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612244934919055:3653] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:03.013957Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:03.014280Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612244934919059:3655], now have 1 active actors on pipe
2025-11-14T16:03:03.014439Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:03.014453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:03.014485Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|e3adbd70-fee7b955-674dcacd-7ec16587_88 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:03.014497Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.014499Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:03.014503Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:03.014504Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.014507Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:03.014518Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:03.014526Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:03.014528Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.014540Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:03.014552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:03.014596Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|e3adbd70-fee7b955-674dcacd-7ec16587_88"
    Status: Active
    SeqNo: 0
  }
}
2025-11-14T16:03:03.014859Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612244934919059:3655] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|e3adbd70-fee7b955-674dcacd-7ec16587_88"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 88
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:03.017121Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:03.017464Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612244934919062:3656], now have 1 active actors on pipe
2025-11-14T16:03:03.017825Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:03.017843Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:03.017863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 88 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:03.017876Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:03.017894Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.017902Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:03.017910Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:03.017912Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.017916Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:03.017941Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 88 partNo 0
2025-11-14T16:03:03.017980Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 88 partNo 0 FormedBlobsCount 0 NewHead: Offset 88 PartNo 0 PackedSize 262214 count 1 nextOffset 89 batches 1
2025-11-14T16:03:03.017993Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:03.017995Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:03.017998Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.018133Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 88,1 HeadOffset 88 endOffset 88 curOffset 89 d0000000000_00000000000000000088_00000_0000000001_00000? size 262204 WTime 1763136183017
2025-11-14T16:03:03.018183Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:03.018201Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 88 partNo 0 count 1 size 262204
2025-11-14T16:03:03.019715Z node 2 :PERSQUEUE DEBUG: partition.cpp:2289: [72075186224037892][Partition][0][StateIdle] Writing. Can't process user action and tx events
2025-11-14T16:03:03.020062Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 88 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.020093Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:03.020111Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:03.020113Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 88 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.020124Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:03.020132Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 88, partNo: 0, Offset: 88 is stored on disk
2025-11-14T16:03:03.020178Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.020186Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.020188Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.020191Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.020193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.020201Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:03.020212Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:03.020274Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:03.020553Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612244934919062:3656] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 88
    Offset: 88
    WriteTimestampMS: 1763136183017
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:03.021226Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:03.021578Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612244934919084:3663], now have 1 active actors on pipe
2025-11-14T16:03:03.021664Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:03.021674Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:03.021705Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|6de52c78-75164386-80ff14fb-ac2c198b_89 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:03.021723Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.021727Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:03.021732Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:03.021734Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.021737Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:03.021748Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:03.021750Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:03.021752Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.021763Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:03.021774Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:03.021900Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:03.022123Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612244934919084:3663] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|6de52c78-75164386-80ff14fb-ac2c198b_89"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|6de52c78-75164386-80ff14fb-ac2c198b_89"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 89
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:03.024337Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:03.024719Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612244934919095:3666], now have 1 active actors on pipe
2025-11-14T16:03:03.025079Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:03.025092Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:03.025109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 89 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:03.025124Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:03.025141Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.025148Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:03.025155Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:03.025162Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.025181Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:03.025207Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 89 partNo 0
2025-11-14T16:03:03.025261Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 89 partNo 0 FormedBlobsCount 0 NewHead: Offset 89 PartNo 0 PackedSize 262214 count 1 nextOffset 90 batches 1
2025-11-14T16:03:03.025272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:03.025274Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:03.025277Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.025385Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 89,1 HeadOffset 89 endOffset 89 curOffset 90 d0000000000_00000000000000000089_00000_0000000001_00000? size 262204 WTime 1763136183025
2025-11-14T16:03:03.025433Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:03.025448Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 89 partNo 0 count 1 size 262204
2025-11-14T16:03:03.027282Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 89 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.027313Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:03.027333Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:03.027342Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 89 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.027346Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:03.027355Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 89, partNo: 0, Offset: 89 is stored on disk
2025-11-14T16:03:03.027392Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.027398Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.027399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.027401Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.027402Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.027408Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:03.027417Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:03.027486Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:03.027748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612244934919095:3666] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 89
    Offset: 89
    WriteTimestampMS: 1763136183024
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:03.028410Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:03.028756Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612244934919098:3667], now have 1 active actors on pipe
2025-11-14T16:03:03.028841Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:03.028858Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:03.028914Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|9d7a355e-929e8be-c8e61ad4-a796b2c9_90 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:03.028928Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.028932Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:03.028937Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:03.028939Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.028944Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:03.028953Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:03.028956Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:03.028960Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.028971Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:03.028986Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:03.029030Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:03.029226Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612244934919098:3667] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|9d7a355e-929e8be-c8e61ad4-a796b2c9_90"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|9d7a355e-929e8be-c8e61ad4-a796b2c9_90"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 90
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:03.031398Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:03.031762Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612244934919101:3668], now have 1 active actors on pipe
2025-11-14T16:03:03.032168Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:03.032183Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:03.032202Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 90 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:03.032217Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:03.032233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.032236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:03.032244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:03.032247Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.032251Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:03.032278Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 90 partNo 0
2025-11-14T16:03:03.032311Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 90 partNo 0 FormedBlobsCount 0 NewHead: Offset 90 PartNo 0 PackedSize 262214 count 1 nextOffset 91 batches 1
2025-11-14T16:03:03.032323Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:03.032326Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:03.032329Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.032450Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 90,1 HeadOffset 90 endOffset 90 curOffset 91 d0000000000_00000000000000000090_00000_0000000001_00000? size 262204 WTime 1763136183031
2025-11-14T16:03:03.032490Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:03.032508Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 90 partNo 0 count 1 size 262204
2025-11-14T16:03:03.034240Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 90 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.034265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:03.034282Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:03.034290Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:03.034304Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 90, partNo: 0, Offset: 90 is stored on disk
2025-11-14T16:03:03.034315Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 90 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.034343Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.034350Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.034352Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.034354Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.034356Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.034363Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:03.034372Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:03.034449Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:03.034748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612244934919101:3668] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 90
    Offset: 90
    WriteTimestampMS: 1763136183031
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:03.035342Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:03.035706Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612244934919105:3670], now have 1 active actors on pipe
2025-11-14T16:03:03.035757Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:03.035767Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:03.035798Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|10928a11-c55112c3-5188ec1-82f46d2c_91 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:03.035810Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.035813Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:03.035817Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:03.035819Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.035823Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:03.035833Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:03.035840Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:03.035843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.035853Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:03.035863Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:03.035920Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|10928a11-c55112c3-5188ec1-82f46d2c_91"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|10928a11-c55112c3-5188ec1-82f46d2c_91"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 91
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:03.037161Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612244934919105:3670] destroyed
2025-11-14T16:03:03.038318Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:03.038696Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612244934919108:3671], now have 1 active actors on pipe
2025-11-14T16:03:03.039109Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:03.039124Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:03.039142Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 91 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:03.039157Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:03.039174Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.039183Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:03.039193Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:03.039195Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.039201Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:03.039231Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 91 partNo 0
2025-11-14T16:03:03.039269Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 91 partNo 0 FormedBlobsCount 0 NewHead: Offset 91 PartNo 0 PackedSize 262214 count 1 nextOffset 92 batches 1
2025-11-14T16:03:03.039283Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:03.039289Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:03.039292Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.039410Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 91,1 HeadOffset 91 endOffset 91 curOffset 92 d0000000000_00000000000000000091_00000_0000000001_00000? size 262204 WTime 1763136183039
2025-11-14T16:03:03.039454Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:03.039469Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 91 partNo 0 count 1 size 262204
2025-11-14T16:03:03.041541Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 91 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.041572Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:03.041586Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 91 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.041592Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:03.041601Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:03.041610Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 91, partNo: 0, Offset: 91 is stored on disk
2025-11-14T16:03:03.041645Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.041657Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.041659Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.041661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.041663Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.041671Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:03.041682Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:03.042266Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:03.042586Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612244934919108:3671] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 91
    Offset: 91
    WriteTimestampMS: 1763136183039
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:03.043296Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:03.043658Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612244934919112:3673], now have 1 active actors on pipe
2025-11-14T16:03:03.043752Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:03.043764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:03.043799Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|bfbfc1e5-c1904b38-f1045a81-36584d58_92 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:03.043811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.043821Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:03.043825Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:03.043827Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.043831Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:03.043841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:03.043844Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:03.043848Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.043862Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:03.043879Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:03.043924Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|bfbfc1e5-c1904b38-f1045a81-36584d58_92"
    Status: Active
    SeqNo: 0
  }
}
2025-11-14T16:03:03.044154Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612244934919112:3673] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|bfbfc1e5-c1904b38-f1045a81-36584d58_92"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 92
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:03.046075Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:03.046453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612244934919115:3674], now have 1 active actors on pipe
2025-11-14T16:03:03.046777Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:03.046793Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:03.046813Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 92 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:03.046828Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:03.046845Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.046852Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:03.046862Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:03.046869Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.046873Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:03.046902Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 92 partNo 0
2025-11-14T16:03:03.046940Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 92 partNo 0 FormedBlobsCount 0 NewHead: Offset 92 PartNo 0 PackedSize 262214 count 1 nextOffset 93 batches 1
2025-11-14T16:03:03.046951Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:03.046954Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:03.046957Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.047073Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 92,1 HeadOffset 92 endOffset 92 curOffset 93 d0000000000_00000000000000000092_00000_0000000001_00000? size 262204 WTime 1763136183046
2025-11-14T16:03:03.047118Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:03.047135Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 92 partNo 0 count 1 size 262204
2025-11-14T16:03:03.048968Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 92 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.048992Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:03.049010Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 92 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.049012Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:03.049020Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:03.049030Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 92, partNo: 0, Offset: 92 is stored on disk
2025-11-14T16:03:03.049058Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.049066Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.049068Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.049071Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.049073Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.049080Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:03.049091Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:03.049181Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
2025-11-14T16:03:03.049453Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612244934919115:3674] destroyed
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 92
    Offset: 92
    WriteTimestampMS: 1763136183046
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:03.050031Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:03.050342Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612244934919119:3676], now have 1 active actors on pipe
2025-11-14T16:03:03.050427Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:03.050441Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:03.050473Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|b03b4db1-12c25b2-87bf865e-6dc273ca_93 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:03.050485Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.050487Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:03.050492Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:03.050494Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.050498Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:03.050506Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:03.050513Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:03.050516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.050537Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:03.050552Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:03.050603Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|b03b4db1-12c25b2-87bf865e-6dc273ca_93"
    Status: Active
    SeqNo: 0
  }
}
2025-11-14T16:03:03.050764Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612244934919119:3676] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|b03b4db1-12c25b2-87bf865e-6dc273ca_93"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 93
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:03.052648Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:03.052983Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612244934919122:3677], now have 1 active actors on pipe
2025-11-14T16:03:03.053329Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:03.053343Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:03.053371Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 93 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:03.053388Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:03.053399Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.053406Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:03.053416Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:03.053423Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.053428Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:03.053454Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 93 partNo 0
2025-11-14T16:03:03.053503Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 93 partNo 0 FormedBlobsCount 0 NewHead: Offset 93 PartNo 0 PackedSize 262214 count 1 nextOffset 94 batches 1
2025-11-14T16:03:03.053516Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:03.053519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:03.053522Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.053631Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 93,1 HeadOffset 93 endOffset 93 curOffset 94 d0000000000_00000000000000000093_00000_0000000001_00000? size 262204 WTime 1763136183053
2025-11-14T16:03:03.053672Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:03.053687Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 93 partNo 0 count 1 size 262204
2025-11-14T16:03:03.055625Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 93 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.055651Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:03.055664Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 93 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.055671Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:03.055679Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:03.055686Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 93, partNo: 0, Offset: 93 is stored on disk
2025-11-14T16:03:03.055718Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.055725Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.055726Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.055730Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.055731Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.055738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:03.055748Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:03.055825Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:03.056081Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612244934919122:3677] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 93
    Offset: 93
    WriteTimestampMS: 1763136183052
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:03.056660Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:03.056991Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612244934919126:3679], now have 1 active actors on pipe
2025-11-14T16:03:03.057066Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:03.057074Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:03.057101Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|d3dc7996-91c1e586-928acc77-ccfdec02_94 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:03.057111Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.057114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:03.057118Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:03.057120Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.057123Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:03.057129Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:03.057135Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:03.057137Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.057143Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:03.057149Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:03.057201Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:03.057385Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612244934919126:3679] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|d3dc7996-91c1e586-928acc77-ccfdec02_94"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|d3dc7996-91c1e586-928acc77-ccfdec02_94"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 94
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:03.059360Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:03.059795Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612244934919129:3680], now have 1 active actors on pipe
2025-11-14T16:03:03.060171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:03.060188Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:03.060210Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 94 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:03.060227Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:03.060244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.060253Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:03.060261Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:03.060267Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.060273Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:03.060304Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 94 partNo 0
2025-11-14T16:03:03.060344Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 94 partNo 0 FormedBlobsCount 0 NewHead: Offset 94 PartNo 0 PackedSize 262214 count 1 nextOffset 95 batches 1
2025-11-14T16:03:03.060357Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:03.060360Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:03.060363Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.060474Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 94,1 HeadOffset 94 endOffset 94 curOffset 95 d0000000000_00000000000000000094_00000_0000000001_00000? size 262204 WTime 1763136183060
2025-11-14T16:03:03.060517Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:03.060533Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 94 partNo 0 count 1 size 262204
2025-11-14T16:03:03.065411Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 94 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.065444Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:03.065465Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:03.065464Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 94 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.065478Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:03.065493Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 94, partNo: 0, Offset: 94 is stored on disk
2025-11-14T16:03:03.065532Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.065540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.065543Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.065546Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.065548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.065559Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:193: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:03.065575Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:03.065654Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 94
    Offset: 94
    WriteTimestampMS: 1763136183060
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 5
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-14T16:03:03.066045Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612244934919129:3680] destroyed
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdGetOwnership {
  }
}
2025-11-14T16:03:03.066676Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:03.067093Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612244934919133:3682], now have 1 active actors on pipe
2025-11-14T16:03:03.067171Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:03.067182Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:03.067214Z node 2 :PERSQUEUE INFO: ownerinfo.cpp:30: new Cookie default|2f87341d-86adcad1-f1e27870-b5a5c4b3_95 generated for partition 0 topic 'rt3.dc1--topic1' owner default
2025-11-14T16:03:03.067236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.067244Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:03.067249Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:03.067252Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.067256Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:03.067265Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:03.067272Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:03.067275Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.067298Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:35: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyOwnerOk. Partition: 0
2025-11-14T16:03:03.067314Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:03.067359Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:03.067532Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612244934919133:3682] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdGetOwnershipResult {
    OwnerCookie: "default|2f87341d-86adcad1-f1e27870-b5a5c4b3_95"
    Status: Active
    SeqNo: 0
  }
}
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  OwnerCookie: "default|2f87341d-86adcad1-f1e27870-b5a5c4b3_95"
  CmdWrite {
    SourceId: "source1"
    SeqNo: 95
    Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:03.072998Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:03.073484Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612244934919136:3683], now have 1 active actors on pipe
2025-11-14T16:03:03.073941Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:03.073956Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:03.073978Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2011: [PQ: 72075186224037892] got client message topic: rt3.dc1--topic1 partition: 0 SourceId: 'source1' SeqNo: 95 partNo : 0 messageNo: 0 size 262144 offset: -1
2025-11-14T16:03:03.073996Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:634: [72075186224037892][Partition][0][StateIdle] Received TPartition::TEvWrite
2025-11-14T16:03:03.074014Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.074023Z node 2 :PERSQUEUE DEBUG: partition.cpp:2407: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxEvent(TMessage)
2025-11-14T16:03:03.074034Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 1, PendingWrites: 0
2025-11-14T16:03:03.074040Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.074047Z node 2 :PERSQUEUE DEBUG: partition.cpp:2471: [72075186224037892][Partition][0][StateIdle] TPartition::ProcessUserActionAndTxPendingCommit(TMessage)
2025-11-14T16:03:03.074075Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1326: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 95 partNo 0
2025-11-14T16:03:03.074116Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1430: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId 'source1' seqNo 95 partNo 0 FormedBlobsCount 0 NewHead: Offset 95 PartNo 0 PackedSize 262214 count 1 nextOffset 96 batches 1
2025-11-14T16:03:03.074127Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 1
2025-11-14T16:03:03.074131Z node 2 :PERSQUEUE DEBUG: partition.cpp:2326: [72075186224037892][Partition][0][StateIdle] Batch completed (1)
2025-11-14T16:03:03.074134Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.074263Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1682: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 95,1 HeadOffset 95 endOffset 95 curOffset 96 d0000000000_00000000000000000095_00000_0000000001_00000? size 262204 WTime 1763136183073
2025-11-14T16:03:03.074305Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:03.074322Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 95 partNo 0 count 1 size 262204
2025-11-14T16:03:03.076568Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 95 count 1 size 262204 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.076595Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:03.076609Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 95 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.076615Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 262151 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:03.076624Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:58: [72075186224037892][Partition][0][StateIdle] TPartition::ReplyWrite. Partition: 0
2025-11-14T16:03:03.076632Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:361: [72075186224037892][Partition][0][StateIdle] Answering for message sourceid: 'source1', Topic: 'rt3.dc1--topic1', Partition: 0, SeqNo: 95, partNo: 0, Offset: 95 is stored on disk
2025-11-14T16:03:03.076661Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.076671Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.076673Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.076676Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.076678Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.076690Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000064_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076694Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000065_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076697Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000066_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076699Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000067_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076702Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000068_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000069_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076706Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000070_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076708Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000071_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076711Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000072_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076714Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000073_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076717Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000074_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076720Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000075_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076723Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000076_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076726Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000077_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076729Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000078_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000079_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076734Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000080_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076738Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000081_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000082_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076743Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000083_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076746Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000084_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000085_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076752Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000086_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000087_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000088_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076761Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000089_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000090_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000091_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076769Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000092_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076772Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000093_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076775Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000094_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076777Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:208: [72075186224037892][Partition][0][StateIdle] Blob key for append d0000000000_00000000000000000095_00000_0000000001_00000? (262204)
2025-11-14T16:03:03.076780Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:214: [72075186224037892][Partition][0][StateIdle] 32 keys were taken away. Let's read 8390528 bytes
2025-11-14T16:03:03.076804Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:03.076820Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:230: [72075186224037892][Partition][0][StateIdle] Begin compaction for 32 blobs
2025-11-14T16:03:03.076823Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000064_00000_0000000001_00000?
2025-11-14T16:03:03.076835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000065_00000_0000000001_00000?
2025-11-14T16:03:03.076847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000066_00000_0000000001_00000?
2025-11-14T16:03:03.076861Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000067_00000_0000000001_00000?
2025-11-14T16:03:03.076876Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000068_00000_0000000001_00000?
2025-11-14T16:03:03.076915Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000069_00000_0000000001_00000?
2025-11-14T16:03:03.076929Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000070_00000_0000000001_00000?
2025-11-14T16:03:03.076942Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000071_00000_0000000001_00000?
2025-11-14T16:03:03.076956Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000072_00000_0000000001_00000?
2025-11-14T16:03:03.076985Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000073_00000_0000000001_00000?
2025-11-14T16:03:03.076998Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000074_00000_0000000001_00000?
2025-11-14T16:03:03.077011Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000075_00000_0000000001_00000?
2025-11-14T16:03:03.077020Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000076_00000_0000000001_00000?
2025-11-14T16:03:03.077036Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000077_00000_0000000001_00000?
2025-11-14T16:03:03.077045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000078_00000_0000000001_00000?
2025-11-14T16:03:03.077054Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000079_00000_0000000001_00000?
2025-11-14T16:03:03.077062Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000080_00000_0000000001_00000?
2025-11-14T16:03:03.077108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000081_00000_0000000001_00000?
2025-11-14T16:03:03.077117Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000082_00000_0000000001_00000?
2025-11-14T16:03:03.077125Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000083_00000_0000000001_00000?
2025-11-14T16:03:03.077133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000084_00000_0000000001_00000?
2025-11-14T16:03:03.077140Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000085_00000_0000000001_00000?
2025-11-14T16:03:03.077148Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000086_00000_0000000001_00000?
2025-11-14T16:03:03.077156Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000087_00000_0000000001_00000?
2025-11-14T16:03:03.077176Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000088_00000_0000000001_00000?
2025-11-14T16:03:03.077184Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000089_00000_0000000001_00000?
2025-11-14T16:03:03.077193Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000090_00000_0000000001_00000?
2025-11-14T16:03:03.077203Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000091_00000_0000000001_00000?
2025-11-14T16:03:03.077212Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000092_00000_0000000001_00000?
2025-11-14T16:03:03.077221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000093_00000_0000000001_00000?
2025-11-14T16:03:03.077229Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000094_00000_0000000001_00000?
2025-11-14T16:03:03.077237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:243: [72075186224037892][Partition][0][StateIdle] Request blob key d0000000000_00000000000000000095_00000_0000000001_00000?
2025-11-14T16:03:03.077246Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:267: [72075186224037892][Partition][0][StateIdle] Request 32 blobs for compaction
2025-11-14T16:03:03.077255Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 64 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:03.077259Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 65 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:03.077263Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 66 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:03.077267Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 67 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:03.077270Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 68 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:03.077273Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 69 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:03.077277Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 70 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:03.077280Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 71 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:03.077284Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 72 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:03.077287Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 73 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:03.077291Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 74 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:03.077294Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 75 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:03.077297Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 76 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:03.077301Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 77 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:03.077307Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 78 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:03.077309Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 79 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:03.077313Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 80 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:03.077315Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 81 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:03.077319Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 82 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:03.077321Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 83 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:03.077325Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 84 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:03.077328Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 85 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:03.077332Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 86 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:03.000000Z
2025-11-14T16:03:03.077335Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 87 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:03.000000Z
2025-11-14T16:03:03.077338Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 88 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:03.000000Z
2025-11-14T16:03:03.077342Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 89 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:03.000000Z
2025-11-14T16:03:03.077345Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 90 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:03.000000Z
2025-11-14T16:03:03.077349Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 91 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:03.000000Z
2025-11-14T16:03:03.077353Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 92 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:03.000000Z
2025-11-14T16:03:03.077356Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 93 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:03.000000Z
2025-11-14T16:03:03.077359Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 94 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:03.000000Z
2025-11-14T16:03:03.077362Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 95 partno 0 count 1 parts_count 0 source 1 size 262204 accessed 0 times before, last time 2025-11-14T16:03:03.000000Z
2025-11-14T16:03:03.077380Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 0. All 32 blobs are from cache.
2025-11-14T16:03:03.077413Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:445: [72075186224037892][Partition][0][StateIdle] Continue blobs compaction
2025-11-14T16:03:03.077421Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[0/32] d0000000000_00000000000000000064_00000_0000000001_00000?
2025-11-14T16:03:03.077424Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.077427Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000064_00000_0000000001_00000?, parameters.CurOffset=64
2025-11-14T16:03:03.077428Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 64 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077433Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 65 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077438Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 66 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077445Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 67 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077449Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 68 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077455Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 69 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077458Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 70 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077474Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 71 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077479Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 72 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077483Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 73 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077486Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 74 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077490Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 75 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077494Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 76 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077497Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 77 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077501Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 78 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077505Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 79 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077509Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 80 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077513Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 81 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077516Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 82 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077519Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 83 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077531Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 84 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077535Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 85 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077541Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 86 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077544Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 87 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077548Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 88 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077551Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 89 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077554Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 90 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077557Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 91 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077563Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 92 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077566Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 93 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.078597Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:03.077569Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 94 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.077573Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 95 partno 0 count 1 parts 0 suffix '63'
2025-11-14T16:03:03.078685Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 64.0/1
2025-11-14T16:03:03.078704Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 64 partNo 0
2025-11-14T16:03:03.078732Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 64 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 262214 count 1 nextOffset 65 batches 1
2025-11-14T16:03:03.078741Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 64:0 appended
2025-11-14T16:03:03.078748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[1/32] d0000000000_00000000000000000065_00000_0000000001_00000?
2025-11-14T16:03:03.078751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.078755Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000065_00000_0000000001_00000?, parameters.CurOffset=65
2025-11-14T16:03:03.078827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 65.0/1
2025-11-14T16:03:03.078848Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 65 partNo 0
CallPersQueueGRPC response:
2025-11-14T16:03:03.079488Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 65 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 524387 count 2 nextOffset 66 batches 1
2025-11-14T16:03:03.079502Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 65:0 appended
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdWriteResult {
    AlreadyWritten: false
    SourceId: "source1"
    SeqNo: 95
    Offset: 95
    WriteTimestampMS: 1763136183073
    PartitionQuotedTimeMs: 0
    TotalTimeInPartitionQueueMs: 0
    WriteTimeMs: 2
    TopicQuotedTimeMs: 0
    WrittenInTx: false
  }
}
2025-11-14T16:03:03.079509Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[2/32] d0000000000_00000000000000000066_00000_0000000001_00000?
2025-11-14T16:03:03.079512Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.079515Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000066_00000_0000000001_00000?, parameters.CurOffset=66
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdRead {
    ClientId: "user1"
    Offset: 0
    Count: 48
    ReadTimestampMs: 0
  }
}
2025-11-14T16:03:03.079631Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 66.0/1
2025-11-14T16:03:03.079642Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 66 partNo 0
2025-11-14T16:03:03.079655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 66 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 786601 count 3 nextOffset 67 batches 2
2025-11-14T16:03:03.079663Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 66:0 appended
2025-11-14T16:03:03.079667Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[3/32] d0000000000_00000000000000000067_00000_0000000001_00000?
2025-11-14T16:03:03.079673Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.079677Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000067_00000_0000000001_00000?, parameters.CurOffset=67
2025-11-14T16:03:03.079730Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 67.0/1
2025-11-14T16:03:03.079740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 67 partNo 0
2025-11-14T16:03:03.080276Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:03.081074Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 67 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 1048774 count 4 nextOffset 68 batches 2
2025-11-14T16:03:03.081088Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 67:0 appended
2025-11-14T16:03:03.081095Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[4/32] d0000000000_00000000000000000068_00000_0000000001_00000?
2025-11-14T16:03:03.081098Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.081103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000068_00000_0000000001_00000?, parameters.CurOffset=68
2025-11-14T16:03:03.081199Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 68.0/1
2025-11-14T16:03:03.081209Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 68 partNo 0
2025-11-14T16:03:03.081221Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 68 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 1310988 count 5 nextOffset 69 batches 3
2025-11-14T16:03:03.081227Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 68:0 appended
2025-11-14T16:03:03.081232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[5/32] d0000000000_00000000000000000069_00000_0000000001_00000?
2025-11-14T16:03:03.081234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.081238Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000069_00000_0000000001_00000?, parameters.CurOffset=69
2025-11-14T16:03:03.081299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 69.0/1
2025-11-14T16:03:03.081306Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 69 partNo 0
2025-11-14T16:03:03.081430Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 69 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 1573161 count 6 nextOffset 70 batches 3
2025-11-14T16:03:03.081441Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 69:0 appended
2025-11-14T16:03:03.081445Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[6/32] d0000000000_00000000000000000070_00000_0000000001_00000?
2025-11-14T16:03:03.081448Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.081450Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000070_00000_0000000001_00000?, parameters.CurOffset=70
2025-11-14T16:03:03.081508Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 70.0/1
2025-11-14T16:03:03.081518Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 70 partNo 0
2025-11-14T16:03:03.081541Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 70 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 1835375 count 7 nextOffset 71 batches 4
2025-11-14T16:03:03.081548Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 70:0 appended
2025-11-14T16:03:03.081552Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[7/32] d0000000000_00000000000000000071_00000_0000000001_00000?
2025-11-14T16:03:03.081553Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.081556Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000071_00000_0000000001_00000?, parameters.CurOffset=71
2025-11-14T16:03:03.081608Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 71.0/1
2025-11-14T16:03:03.081616Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 71 partNo 0
2025-11-14T16:03:03.081728Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 71 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 2097548 count 8 nextOffset 72 batches 4
2025-11-14T16:03:03.081740Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 71:0 appended
2025-11-14T16:03:03.081746Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[8/32] d0000000000_00000000000000000072_00000_0000000001_00000?
2025-11-14T16:03:03.081748Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.081751Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000072_00000_0000000001_00000?, parameters.CurOffset=72
2025-11-14T16:03:03.081807Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 72.0/1
2025-11-14T16:03:03.081811Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 72 partNo 0
2025-11-14T16:03:03.081819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 72 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 2359762 count 9 nextOffset 73 batches 5
2025-11-14T16:03:03.081822Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 72:0 appended
2025-11-14T16:03:03.081825Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[9/32] d0000000000_00000000000000000073_00000_0000000001_00000?
2025-11-14T16:03:03.081827Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.081830Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000073_00000_0000000001_00000?, parameters.CurOffset=73
2025-11-14T16:03:03.081881Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 73.0/1
2025-11-14T16:03:03.081885Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 73 partNo 0
2025-11-14T16:03:03.081994Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 73 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 2621935 count 10 nextOffset 74 batches 5
2025-11-14T16:03:03.081997Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 73:0 appended
2025-11-14T16:03:03.082001Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[10/32] d0000000000_00000000000000000074_00000_0000000001_00000?
2025-11-14T16:03:03.082003Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.082005Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000074_00000_0000000001_00000?, parameters.CurOffset=74
2025-11-14T16:03:03.082063Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 74.0/1
2025-11-14T16:03:03.082067Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 74 partNo 0
2025-11-14T16:03:03.082076Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 74 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 2884149 count 11 nextOffset 75 batches 6
2025-11-14T16:03:03.082078Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 74:0 appended
2025-11-14T16:03:03.082081Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[11/32] d0000000000_00000000000000000075_00000_0000000001_00000?
2025-11-14T16:03:03.082083Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.082085Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000075_00000_0000000001_00000?, parameters.CurOffset=75
2025-11-14T16:03:03.082139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 75.0/1
2025-11-14T16:03:03.082143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 75 partNo 0
2025-11-14T16:03:03.082251Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 75 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 3146322 count 12 nextOffset 76 batches 6
2025-11-14T16:03:03.082254Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 75:0 appended
2025-11-14T16:03:03.082258Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[12/32] d0000000000_00000000000000000076_00000_0000000001_00000?
2025-11-14T16:03:03.082260Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.082262Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000076_00000_0000000001_00000?, parameters.CurOffset=76
2025-11-14T16:03:03.082317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 76.0/1
2025-11-14T16:03:03.082321Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 76 partNo 0
2025-11-14T16:03:03.082329Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 76 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 3408536 count 13 nextOffset 77 batches 7
2025-11-14T16:03:03.082331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 76:0 appended
2025-11-14T16:03:03.082334Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[13/32] d0000000000_00000000000000000077_00000_0000000001_00000?
2025-11-14T16:03:03.082336Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.082339Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000077_00000_0000000001_00000?, parameters.CurOffset=77
2025-11-14T16:03:03.082389Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 77.0/1
2025-11-14T16:03:03.082392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 77 partNo 0
2025-11-14T16:03:03.082498Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 77 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 3670709 count 14 nextOffset 78 batches 7
2025-11-14T16:03:03.082501Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 77:0 appended
2025-11-14T16:03:03.082505Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[14/32] d0000000000_00000000000000000078_00000_0000000001_00000?
2025-11-14T16:03:03.082507Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.082510Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000078_00000_0000000001_00000?, parameters.CurOffset=78
2025-11-14T16:03:03.082576Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 78.0/1
2025-11-14T16:03:03.082580Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 78 partNo 0
2025-11-14T16:03:03.082588Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 78 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 3932923 count 15 nextOffset 79 batches 8
2025-11-14T16:03:03.082591Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 78:0 appended
2025-11-14T16:03:03.082595Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[15/32] d0000000000_00000000000000000079_00000_0000000001_00000?
2025-11-14T16:03:03.082597Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.082599Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000079_00000_0000000001_00000?, parameters.CurOffset=79
2025-11-14T16:03:03.082652Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 79.0/1
2025-11-14T16:03:03.082655Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 79 partNo 0
2025-11-14T16:03:03.082763Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 79 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 4195096 count 16 nextOffset 80 batches 8
2025-11-14T16:03:03.082766Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 79:0 appended
2025-11-14T16:03:03.082770Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[16/32] d0000000000_00000000000000000080_00000_0000000001_00000?
2025-11-14T16:03:03.082771Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.082774Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000080_00000_0000000001_00000?, parameters.CurOffset=80
2025-11-14T16:03:03.082831Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 80.0/1
2025-11-14T16:03:03.082835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 80 partNo 0
2025-11-14T16:03:03.082845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 80 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 4457310 count 17 nextOffset 81 batches 9
2025-11-14T16:03:03.082847Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 80:0 appended
2025-11-14T16:03:03.082850Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[17/32] d0000000000_00000000000000000081_00000_0000000001_00000?
2025-11-14T16:03:03.082852Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.082854Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000081_00000_0000000001_00000?, parameters.CurOffset=81
2025-11-14T16:03:03.082909Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 81.0/1
2025-11-14T16:03:03.082913Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 81 partNo 0
2025-11-14T16:03:03.083022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 81 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 4719483 count 18 nextOffset 82 batches 9
2025-11-14T16:03:03.083025Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 81:0 appended
2025-11-14T16:03:03.083029Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[18/32] d0000000000_00000000000000000082_00000_0000000001_00000?
2025-11-14T16:03:03.083033Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.083037Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000082_00000_0000000001_00000?, parameters.CurOffset=82
2025-11-14T16:03:03.083108Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 82.0/1
2025-11-14T16:03:03.083111Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 82 partNo 0
2025-11-14T16:03:03.083120Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 82 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 4981697 count 19 nextOffset 83 batches 10
2025-11-14T16:03:03.083122Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 82:0 appended
2025-11-14T16:03:03.083126Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[19/32] d0000000000_00000000000000000083_00000_0000000001_00000?
2025-11-14T16:03:03.083128Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.083131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000083_00000_0000000001_00000?, parameters.CurOffset=83
2025-11-14T16:03:03.083195Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 83.0/1
2025-11-14T16:03:03.083198Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 83 partNo 0
2025-11-14T16:03:03.083304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 83 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 5243870 count 20 nextOffset 84 batches 10
2025-11-14T16:03:03.083307Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 83:0 appended
2025-11-14T16:03:03.083312Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[20/32] d0000000000_00000000000000000084_00000_0000000001_00000?
2025-11-14T16:03:03.083314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.083317Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000084_00000_0000000001_00000?, parameters.CurOffset=84
2025-11-14T16:03:03.083374Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 84.0/1
2025-11-14T16:03:03.083377Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 84 partNo 0
2025-11-14T16:03:03.083385Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 84 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 5506084 count 21 nextOffset 85 batches 11
2025-11-14T16:03:03.083388Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 84:0 appended
2025-11-14T16:03:03.083392Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[21/32] d0000000000_00000000000000000085_00000_0000000001_00000?
2025-11-14T16:03:03.083394Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.083399Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000085_00000_0000000001_00000?, parameters.CurOffset=85
2025-11-14T16:03:03.083461Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 85.0/1
2025-11-14T16:03:03.083465Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 85 partNo 0
2025-11-14T16:03:03.084031Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 85 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 5768257 count 22 nextOffset 86 batches 11
2025-11-14T16:03:03.084035Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 85:0 appended
2025-11-14T16:03:03.084041Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[22/32] d0000000000_00000000000000000086_00000_0000000001_00000?
2025-11-14T16:03:03.084044Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.084047Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000086_00000_0000000001_00000?, parameters.CurOffset=86
2025-11-14T16:03:03.084107Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 86.0/1
2025-11-14T16:03:03.084112Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 86 partNo 0
2025-11-14T16:03:03.084131Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 86 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 6030471 count 23 nextOffset 87 batches 12
2025-11-14T16:03:03.084133Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 86:0 appended
2025-11-14T16:03:03.084137Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[23/32] d0000000000_00000000000000000087_00000_0000000001_00000?
2025-11-14T16:03:03.084139Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.084143Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000087_00000_0000000001_00000?, parameters.CurOffset=87
2025-11-14T16:03:03.084196Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 87.0/1
2025-11-14T16:03:03.084200Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 87 partNo 0
2025-11-14T16:03:03.084744Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 87 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 6292644 count 24 nextOffset 88 batches 12
2025-11-14T16:03:03.084749Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 87:0 appended
2025-11-14T16:03:03.084754Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[24/32] d0000000000_00000000000000000088_00000_0000000001_00000?
2025-11-14T16:03:03.084756Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.084760Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000088_00000_0000000001_00000?, parameters.CurOffset=88
2025-11-14T16:03:03.084819Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 88.0/1
2025-11-14T16:03:03.084824Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 88 partNo 0
2025-11-14T16:03:03.084833Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 88 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 6554858 count 25 nextOffset 89 batches 13
2025-11-14T16:03:03.084835Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 88:0 appended
2025-11-14T16:03:03.084839Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[25/32] d0000000000_00000000000000000089_00000_0000000001_00000?
2025-11-14T16:03:03.084841Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.084845Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000089_00000_0000000001_00000?, parameters.CurOffset=89
2025-11-14T16:03:03.084896Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 89.0/1
2025-11-14T16:03:03.085571Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 89 partNo 0
2025-11-14T16:03:03.086134Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 89 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 6817031 count 26 nextOffset 90 batches 13
2025-11-14T16:03:03.086149Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 89:0 appended
2025-11-14T16:03:03.086157Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[26/32] d0000000000_00000000000000000090_00000_0000000001_00000?
2025-11-14T16:03:03.086159Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.086162Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000090_00000_0000000001_00000?, parameters.CurOffset=90
2025-11-14T16:03:03.086219Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 90.0/1
2025-11-14T16:03:03.086224Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 90 partNo 0
2025-11-14T16:03:03.086234Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 90 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 7079245 count 27 nextOffset 91 batches 14
2025-11-14T16:03:03.086237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 90:0 appended
2025-11-14T16:03:03.086241Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[27/32] d0000000000_00000000000000000091_00000_0000000001_00000?
2025-11-14T16:03:03.086243Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.086247Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000091_00000_0000000001_00000?, parameters.CurOffset=91
2025-11-14T16:03:03.086299Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 91.0/1
2025-11-14T16:03:03.086304Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 91 partNo 0
2025-11-14T16:03:03.086948Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 91 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 7341418 count 28 nextOffset 92 batches 14
2025-11-14T16:03:03.086951Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 91:0 appended
2025-11-14T16:03:03.086957Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[28/32] d0000000000_00000000000000000092_00000_0000000001_00000?
2025-11-14T16:03:03.086960Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.086963Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000092_00000_0000000001_00000?, parameters.CurOffset=92
2025-11-14T16:03:03.087022Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 92.0/1
2025-11-14T16:03:03.087026Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 92 partNo 0
2025-11-14T16:03:03.087037Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 92 partNo 0 FormedBlobsCount 0 NewHead: Offset 64 PartNo 0 PackedSize 7603632 count 29 nextOffset 93 batches 15
2025-11-14T16:03:03.087039Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 92:0 appended
2025-11-14T16:03:03.087043Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[29/32] d0000000000_00000000000000000093_00000_0000000001_00000?
2025-11-14T16:03:03.087045Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.087048Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000093_00000_0000000001_00000?, parameters.CurOffset=93
2025-11-14T16:03:03.087099Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 93.0/1
2025-11-14T16:03:03.087103Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 93 partNo 0
2025-11-14T16:03:03.095653Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:93: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob sourceId '' seqNo 93 partNo 0 result is x0000000000_00000000000000000062_00000_0000000031_00000 size 8128009
2025-11-14T16:03:03.095689Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:1104: [72075186224037892][Partition][0][StateIdle] writing blob: topic 'rt3.dc1--topic1' partition 0 old key x0000000000_00000000000000000062_00000_0000000031_00000 new key d0000000000_00000000000000000062_00000_0000000031_00000 size 8128009 WTime 1763136183095
2025-11-14T16:03:03.095733Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 93 partNo 0 FormedBlobsCount 1 NewHead: Offset 93 PartNo 0 PackedSize 262214 count 1 nextOffset 94 batches 1
2025-11-14T16:03:03.095742Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 93:0 appended
2025-11-14T16:03:03.095758Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[30/32] d0000000000_00000000000000000094_00000_0000000001_00000?
2025-11-14T16:03:03.095762Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.095767Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000094_00000_0000000001_00000?, parameters.CurOffset=94
2025-11-14T16:03:03.095849Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 94.0/1
2025-11-14T16:03:03.095866Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 94 partNo 0
2025-11-14T16:03:03.096232Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 94 partNo 0 FormedBlobsCount 0 NewHead: Offset 93 PartNo 0 PackedSize 524387 count 2 nextOffset 95 batches 1
2025-11-14T16:03:03.096237Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 94:0 appended
2025-11-14T16:03:03.096244Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:478: [72075186224037892][Partition][0][StateIdle] key[31/32] d0000000000_00000000000000000095_00000_0000000001_00000?
2025-11-14T16:03:03.096245Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:508: [72075186224037892][Partition][0][StateIdle] Need to compact head 0
2025-11-14T16:03:03.096249Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:278: [72075186224037892][Partition][0][StateIdle] requestedBlob.Key=d0000000000_00000000000000000095_00000_0000000001_00000?, parameters.CurOffset=95
2025-11-14T16:03:03.096314Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:287: [72075186224037892][Partition][0][StateIdle] Try append part 95.0/1
2025-11-14T16:03:03.096319Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:58: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob processing sourceId 'source1' seqNo 95 partNo 0
2025-11-14T16:03:03.096331Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:136: [72075186224037892][Partition][0][StateIdle] Topic 'rt3.dc1--topic1' partition 0 part blob complete sourceId '' seqNo 95 partNo 0 FormedBlobsCount 0 NewHead: Offset 93 PartNo 0 PackedSize 786601 count 3 nextOffset 96 batches 2
2025-11-14T16:03:03.096333Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:367: [72075186224037892][Partition][0][StateIdle] Part 95:0 appended
2025-11-14T16:03:03.096856Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:637: [72075186224037892][Partition][0][StateIdle] Add new write blob: topic 'rt3.dc1--topic1' partition 0 compactOffset 93,3 HeadOffset 62 endOffset 64 curOffset 96 d0000000000_00000000000000000093_00000_0000000003_00000| size 786591 WTime 1763136183095
2025-11-14T16:03:03.097689Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612244934919136:3683] destroyed
2025-11-14T16:03:03.097698Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612244934919140:3685], now have 1 active actors on pipe
2025-11-14T16:03:03.097735Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:03.097743Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:03.097760Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:03.097773Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 62 partNo 0 count 31 size 8128009
2025-11-14T16:03:03.097781Z node 2 :PERSQUEUE DEBUG: read.h:313: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough blob. Partition 0 offset 93 partNo 0 count 3 size 786591
2025-11-14T16:03:03.098236Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 4 Topic 'rt3.dc1--topic1' partition 0 user user1 offset 0 partno 0 count 48 size 26214400 endOffset 96 max time lag 0ms effective offset 0
2025-11-14T16:03:03.098453Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 4 added 2 blobs, size 16256018 count 63 last offset 31, current partition end offset: 96
2025-11-14T16:03:03.098457Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 4. Send blob request.
2025-11-14T16:03:03.098474Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 31 parts_count 0 source 1 size 8128009 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:03.098479Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 31 partno 0 count 31 parts_count 0 source 1 size 8128009 accessed 0 times before, last time 2025-11-14T16:03:02.000000Z
2025-11-14T16:03:03.098489Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 4. All 2 blobs are from cache.
2025-11-14T16:03:03.098512Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 31 parts 0 suffix '0'
2025-11-14T16:03:03.098518Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 31 parts 0 suffix '0'
2025-11-14T16:03:03.098522Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 2 blobs
2025-11-14T16:03:03.098734Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.099286Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 2 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.099792Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 4 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.100316Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 6 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.100850Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 8 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.101467Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 10 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.102015Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 12 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.102376Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 14 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.102838Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 16 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.103198Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 18 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.103632Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 20 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.104242Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 22 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.106457Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 24 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.106980Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 26 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.107452Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 28 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.107538Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 30 totakecount 31 count 1 size 262182 from pos 0 cbcount 1
2025-11-14T16:03:03.107618Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 31 totakecount 31 count 1 size 262182 from pos 0 cbcount 1
2025-11-14T16:03:03.107770Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 32 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.107953Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 34 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.108256Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 36 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.108549Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 38 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.108862Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 40 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.109223Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 42 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.109510Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 44 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.109919Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 46 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.110047Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:03.120087Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.120107Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.120110Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.120114Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.120121Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.123987Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 62 count 31 size 8128009 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.124009Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:321: Caching head blob in L1. Partition 0 offset 93 count 3 size 786591 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.124029Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:03.124058Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 31 parts 0 suffix '0' size 8128009
2025-11-14T16:03:03.124074Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 62 partno 0 count 2 parts 0 suffix '124' size 524387
2025-11-14T16:03:03.124078Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:123: PQ Cache (L2). Evicting blob. Tablet '72075186224037892' partition 0 offset 64 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.124086Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:129: PQ Cache (L2). Adding blob. Tablet '72075186224037892' partition 0 offset 93 partno 0 count 3 parts 0 suffix '124' size 786591
2025-11-14T16:03:03.124228Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.124242Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.124245Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.124248Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.124250Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.124293Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:03.124306Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 64 size 262204 cause it's been evicted from L2. Actual L1 size: 36
2025-11-14T16:03:03.124308Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:438: Erasing blob in L1. Partition 0 offset 62 size 524387 cause it's been evicted from L2. Actual L1 size: 35
2025-11-14T16:03:03.124316Z node 2 :PERSQUEUE NOTICE: read.h:372: [72075186224037892][PQCacheProxy]Have to remove new data from cache. Topic rt3.dc1--topic1, tablet id72075186224037892, cookie 0
2025-11-14T16:03:03.124321Z node 2 :PERSQUEUE DEBUG: read.h:275: [72075186224037892][PQCacheProxy]CacheProxy. Passthrough write request to KV
2025-11-14T16:03:03.124333Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000064_00000_0000000001_00000?(+) to d0000000000_00000000000000000064_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124335Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000065_00000_0000000001_00000?(+) to d0000000000_00000000000000000065_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124336Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000066_00000_0000000001_00000?(+) to d0000000000_00000000000000000066_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124338Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000067_00000_0000000001_00000?(+) to d0000000000_00000000000000000067_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124339Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000068_00000_0000000001_00000?(+) to d0000000000_00000000000000000068_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124341Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000069_00000_0000000001_00000?(+) to d0000000000_00000000000000000069_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124343Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000070_00000_0000000001_00000?(+) to d0000000000_00000000000000000070_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124344Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000071_00000_0000000001_00000?(+) to d0000000000_00000000000000000071_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124346Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000072_00000_0000000001_00000?(+) to d0000000000_00000000000000000072_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124347Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000073_00000_0000000001_00000?(+) to d0000000000_00000000000000000073_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124350Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000074_00000_0000000001_00000?(+) to d0000000000_00000000000000000074_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124351Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000075_00000_0000000001_00000?(+) to d0000000000_00000000000000000075_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124352Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000076_00000_0000000001_00000?(+) to d0000000000_00000000000000000076_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124354Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000077_00000_0000000001_00000?(+) to d0000000000_00000000000000000077_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124355Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000078_00000_0000000001_00000?(+) to d0000000000_00000000000000000078_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124357Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000079_00000_0000000001_00000?(+) to d0000000000_00000000000000000079_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124359Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000080_00000_0000000001_00000?(+) to d0000000000_00000000000000000080_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124361Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000081_00000_0000000001_00000?(+) to d0000000000_00000000000000000081_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124362Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000082_00000_0000000001_00000?(+) to d0000000000_00000000000000000082_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124363Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000083_00000_0000000001_00000?(+) to d0000000000_00000000000000000083_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124365Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000084_00000_0000000001_00000?(+) to d0000000000_00000000000000000084_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124366Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000085_00000_0000000001_00000?(+) to d0000000000_00000000000000000085_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124367Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000086_00000_0000000001_00000?(+) to d0000000000_00000000000000000086_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124369Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000087_00000_0000000001_00000?(+) to d0000000000_00000000000000000087_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124370Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000088_00000_0000000001_00000?(+) to d0000000000_00000000000000000088_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124371Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000089_00000_0000000001_00000?(+) to d0000000000_00000000000000000089_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124372Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000090_00000_0000000001_00000?(+) to d0000000000_00000000000000000090_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124374Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000091_00000_0000000001_00000?(+) to d0000000000_00000000000000000091_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124375Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000092_00000_0000000001_00000?(+) to d0000000000_00000000000000000092_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124377Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000093_00000_0000000001_00000?(+) to d0000000000_00000000000000000093_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124380Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000094_00000_0000000001_00000?(+) to d0000000000_00000000000000000094_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124381Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000095_00000_0000000001_00000?(+) to d0000000000_00000000000000000095_00000_0000000001_00000?(+)
2025-11-14T16:03:03.124383Z node 2 :PERSQUEUE DEBUG: read.h:350: [72075186224037892][PQCacheProxy]CacheProxy. Delete blobs from d0000000000_00000000000000000062_00000_0000000002_00000|(+) to d0000000000_00000000000000000062_00000_0000000002_00000|(+)
2025-11-14T16:03:03.125494Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 65 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125507Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 66 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125512Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 67 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125516Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 68 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125523Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 69 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125527Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 70 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125531Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 71 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125535Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 72 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125540Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 73 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125545Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 74 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125548Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 75 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125552Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 76 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125557Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 77 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125560Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 78 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125564Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 79 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125567Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 80 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125573Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 81 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125580Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 82 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125588Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 83 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125591Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 84 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125596Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 85 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125600Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 86 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125603Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 87 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125607Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 88 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125610Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 89 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125613Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 90 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125617Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 91 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125621Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 92 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125625Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 93 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125628Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 94 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125631Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:371: Deleting head blob in L1. Partition 0 offset 95 count 1 actorID [2:7572612241123374837:2364]
2025-11-14T16:03:03.125666Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 65 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125668Z node 2 :PERSQUEUE DEBUG: partition.cpp:2137: [72075186224037892][Partition][0][StateIdle] Received TEvKeyValue::TEvResponse
2025-11-14T16:03:03.125677Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 66 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125680Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 67 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125683Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 68 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125685Z node 2 :PERSQUEUE DEBUG: partition_write.cpp:570: [72075186224037892][Partition][0][StateIdle] TPartition::HandleWriteResponse writeNewSize# 0 WriteNewSizeFromSupportivePartitions# 0
2025-11-14T16:03:03.125686Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 69 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125688Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 70 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125691Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 71 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125694Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 72 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125696Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.125697Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 73 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125699Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.125700Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 74 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125701Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.125703Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 75 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125703Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.125706Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 76 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125706Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.125708Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 77 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125711Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 78 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125712Z node 2 :PERSQUEUE DEBUG: partition_compaction.cpp:183: [72075186224037892][Partition][0][StateIdle] No data for blobs compaction
2025-11-14T16:03:03.125714Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 79 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125718Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 80 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125720Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 81 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125723Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 82 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125726Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 83 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125728Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 84 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125731Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 85 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125733Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 86 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125736Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 87 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125740Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 88 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125742Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 89 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125744Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 90 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125747Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 91 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125749Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 92 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125752Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 93 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125755Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 94 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.125757Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:155: PQ Cache (L2). Removed. Tablet '72075186224037892' partition 0 offset 95 partno 0 count 1 parts 0 suffix '63' size 262204
2025-11-14T16:03:03.127808Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:03.141335Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612244934919140:3685] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdReadResult {
    MaxOffset: 96
    Result {
      Offset: 0
      Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
CallPersQueueGRPC request to localhost:2468
PartitionRequest {
  Topic: "rt3.dc1--topic1"
  Partition: 0
  CmdRead {
    ClientId: "user1"
    Offset: 0
    Count: 48
    ReadTimestampMs: 0
  }
}
2025-11-14T16:03:03.208399Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1365: proxy got request <none> IsMetaRequest 0 IsFetchRequest 0
2025-11-14T16:03:03.209119Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2721: [PQ: 72075186224037892] server connected, pipe [1:7572612244934919159:3696], now have 1 active actors on pipe
2025-11-14T16:03:03.209384Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:146: Handle TEvRequest topic: 'rt3.dc1--topic1' requestId: <none>
2025-11-14T16:03:03.209395Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2635: [PQ: 72075186224037892] got client message batch for topic 'rt3.dc1--topic1' partition 0
2025-11-14T16:03:03.209441Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:887: [72075186224037892][Partition][0][StateIdle] read cookie 5 Topic 'rt3.dc1--topic1' partition 0 user user1 offset 0 partno 0 count 48 size 26214400 endOffset 96 max time lag 0ms effective offset 0
2025-11-14T16:03:03.209473Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1112: [72075186224037892][Partition][0][StateIdle] read cookie 5 added 2 blobs, size 16256018 count 62 last offset 31, current partition end offset: 96
2025-11-14T16:03:03.209476Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:1128: [72075186224037892][Partition][0][StateIdle] Reading cookie 5. Send blob request.
2025-11-14T16:03:03.209489Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 0 partno 0 count 31 parts_count 0 source 1 size 8128009 accessed 1 times before, last time 2025-11-14T16:03:03.000000Z
2025-11-14T16:03:03.209492Z node 2 :PERSQUEUE DEBUG: cache_eviction.h:497: Got data from cache. Partition 0 offset 31 partno 0 count 31 parts_count 0 source 1 size 8128009 accessed 1 times before, last time 2025-11-14T16:03:03.000000Z
2025-11-14T16:03:03.209502Z node 2 :PERSQUEUE DEBUG: read.h:126: [72075186224037892][PQCacheProxy]Reading cookie 5. All 2 blobs are from cache.
2025-11-14T16:03:03.209528Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:576: FormAnswer for 2 blobs
2025-11-14T16:03:03.209804Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 0 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.210010Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 2 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.210231Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 4 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.210406Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 6 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.210586Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 8 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.210776Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 10 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.210955Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 12 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.211137Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 14 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.211229Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 0 partno 0 count 31 parts 0 suffix '0'
2025-11-14T16:03:03.211235Z node 2 :PERSQUEUE DEBUG: pq_l2_cache.cpp:201: PQ Cache (L2). Touched. Tablet '72075186224037892' partition 0 offset 31 partno 0 count 31 parts 0 suffix '0'
2025-11-14T16:03:03.211343Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 16 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.211538Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 18 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.211733Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 20 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.211922Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 22 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.212116Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 24 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.212307Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 26 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.212499Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 28 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.212574Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 30 totakecount 31 count 1 size 262182 from pos 0 cbcount 1
2025-11-14T16:03:03.212700Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 31 totakecount 31 count 1 size 262182 from pos 0 cbcount 1
2025-11-14T16:03:03.212886Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 32 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.213244Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 34 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.213463Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 36 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.213666Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 38 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.214221Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 40 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.214771Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 42 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.215322Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 44 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.215868Z node 2 :PERSQUEUE DEBUG: partition_read.cpp:496: FormAnswer processing batch offset 46 totakecount 31 count 2 size 524365 from pos 0 cbcount 2
2025-11-14T16:03:03.215976Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:181: Answer ok topic: 'rt3.dc1--topic1' partition: 0 messageNo: 0  requestId: <none> cookie: 0
2025-11-14T16:03:03.220519Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.220540Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.220544Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.220548Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.220551Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.233942Z node 1 :PERSQUEUE INFO: msgbus_server_persqueue.cpp:1524: proxy answer <none>
2025-11-14T16:03:03.240458Z node 2 :PERSQUEUE DEBUG: pq_impl.cpp:2746: [PQ: 72075186224037892] server disconnected, pipe [1:7572612244934919159:3696] destroyed
CallPersQueueGRPC response:
Status: 1
ErrorCode: OK
PartitionResponse {
  CmdReadResult {
    MaxOffset: 96
    Result {
      Offset: 0
      Data: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...
(TRUNCATED)
2025-11-14T16:03:03.320811Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.320834Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.320837Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.320841Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.320843Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
2025-11-14T16:03:03.421203Z node 2 :PERSQUEUE DEBUG: partition.cpp:2313: [72075186224037892][Partition][0][StateIdle] Process user action and tx events
2025-11-14T16:03:03.421226Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.421229Z node 2 :PERSQUEUE DEBUG: partition.cpp:2321: [72075186224037892][Partition][0][StateIdle] Process user action and tx pending commits
2025-11-14T16:03:03.421233Z node 2 :PERSQUEUE DEBUG: partition.cpp:2372: [72075186224037892][Partition][0][StateIdle] Events: 0, PendingCommits: 0, PendingWrites: 0
2025-11-14T16:03:03.421236Z node 2 :PERSQUEUE DEBUG: partition.cpp:2331: [72075186224037892][Partition][0][StateIdle] Try persist
[[bad]]assertion failed at ydb/services/persqueue_v1/persqueue_ut.cpp:3344, virtual void NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TTestCaseCacheHead::Execute_(NUnitTest::TTestContext &): (info_half1.BlobsFromDisk == 1) failed: (0 != 1) [[rst]]
[[alt1]]TBackTrace::Capture()+28 (0x14F5D62C)
NUnitTest::NPrivate::RaiseError(char const*, TBasicString<char, std::__y1::char_traits<char>> const&, bool)+128 (0x151197B0)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TTestCaseCacheHead::Execute_(NUnitTest::TTestContext&)+5702 (0x14C92426)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TCurrentTest::Execute()::'lambda'()::operator()() const+71 (0x14D732D7)
NUnitTest::TTestBase::Run(std::__y1::function<void ()>, TBasicString<char, std::__y1::char_traits<char>> const&, char const*, bool)+120 (0x1511B988)
NKikimr::NPersQueueTests::NTestSuiteTPersQueueTest::TCurrentTest::Execute()+436 (0x14D72BD4)
NUnitTest::TTestFactory::Execute()+817 (0x1511C0D1)
NUnitTest::RunMain(int, char**)+3229 (0x1512E53D)
??+0 (0x7F8CBDA92D90)
__libc_start_main+128 (0x7F8CBDA92E40)
_start+41 (0x13D70029)
[[rst]]
